Botanicus Blog

Image download problem fixed

2009-01-27T15:39:00.003-06:00

Lately, many Botanicus users have been experiencing difficulty in downloading individual images from the site. This has been happening particularly when users have attempted to download the high quality versions of the images.

We wanted to inform you the users that the problem has now been fixed, and you should not have any further trouble in saving images from Botanicus. However, if you do still come across this problem while using the site, please let us know.

We apologize for the inconvenience this may have caused anyone, and we greatly appreciate your patience while we resolved the issue.

Mike Blomberg
Imaging Lab Coordinator
Missouri Botanical Garden

What is a JP2 file?

2009-01-23T15:50:00.005-06:00

"What is a JP2 file? And what do I do with it after I download it from your site?"

These two questions arise frequently with users of Botanicus, so we thought we would make a blog entry to address the topic.

JP2 is the file extension for the JPEG2000 image file format. Think of it as the next generation of the more commonly known JPEG file format. It was introduced in the year 2000 (hence the name), and in principal, it's very similar to JPEG in that they are both compressed image file formats. Because of the compressed file size, JPEG's have long been one of the web standards for displaying images and graphics. The major downfall to JPEG images is that the compression has a negative effect on image quality. The more an image is compressed when saving a JPEG, the smaller the file size will be but also the more the quality deteriorates. JPEG2000 files on the other hand can be compressed to much smaller file sizes with far less deterioration in image quality, which makes it an ideal format for us to serve images to the users of Botanicus.

JP2 files, however, have a disadvantage of their own. Many software authors have failed to embrace this new format yet, and thus programs have been slow in development to offer the ability to open the files. This creates a bit of a challenge for Botanicus users who want to view the files locally on their computer after downloading... Luckily there are four options we recommend for opening JP2 files (three of which are free):

IrfanView - A freeware image viewer with limited image editing capabilities.
Adobe Photoshop - Current versions offer support for JP2 files, however you will need a plug-in to open JP2 files in older versions.
MrSID Decode - A free utility available for download from Lizardtech that allows for the decompression of JP2 files to TIF files.
ExpressView browser plug-in - A free downloadable plug-in also available from Lizardtech for standard browsers (including Safari for Mac users) that allows JP2 files to be resaved to other more readily usable formats (JPEG, TIF, etc.).

There are numerous other options available as well (especially as support for JPEG2000 files continues to grow). For more information on other programs that can open JP2 files, please visit the "Application support" section of the Wikipedia entry for JPEG2000. There you will find an extensive list of other popular programs that currently support the JPEG2000 file format.

Mike Blomberg
Imaging Lab Coordinator
Missouri Botanical Garden

One million pages!

2008-10-08T15:09:00.004-05:00

Today we passed an important milestone in the life of Botanicus. Overnight we published our one millionth page!

It has been a pretty amazing journey getting to one million pages. Botanicus started years ago with a vision and humble beginnings. Scanning for Botanicus started with one single black and white Indus 5001 scanner and somewhat of a makeshift database in which basic information was entered about the books being scanned. Since then Botanicus has grown and changed in ways that even we never imagined that it would! The procedure of getting a book online is now a streamlined series of events that requires little action on behalf of the imaging, library, or information technology (IT) staffs. Thanks to the efforts and innovations from our staff members in these various departments, this streamlining has increased our output well beyond what we thought was possible years ago at the beginning of the project! Our staff that is actively working on Botanicus has grown and currently includes six full-time imaging technicians, two library staff members, and two members of the IT department as well with the assistance of countless others in these departments. We have five full-color book scanning stations (three Indus 5002 scanners, one Kodak i280 sheet feed scanner, and a Better Light copy stand setup) that lately have all been running full shifts five days a week. This is certainly a far cry from our humble beginnings of that single black and white Indus 5001 scanner being used for Botanicus scanning!

Even the Botanicus website itself has evolved into a robust, user-friendly wealth of knowledge… The feedback we have received from the users of the site has proven to be an invaluable resource and has helped to shape the development of the site. Of course, we expected the site to be helpful to people around the world, but again, our expectations have been far surpassed at this point in time! Excitingly, our efforts are no longer limited strictly to just our own website. For example, we now have thriving contributing partnerships in which we share our imaged material with organizations such as the Biodiversity Heritage Library (BHL) and the Encyclopedia of Life. The Botanicus site was always intended to serve the public in general as a free resource of botanical information, but it is interesting to see not only the user hit counts but also the demographics of the site continue to grow over the years. Users of Botanicus are not limited to just botanists and come from all walks of life (artists, historians, medical researchers, etc.). The expanse and impact that Botanicus has had and continues to have is nothing short of astonishing!

Many thanks are due to everyone involved in this project. This includes the library staff, the imaging technicians, and the people in our information technology department who have worked on the Botanicus project over the years. Without the diligent efforts and hard work of these staff members, Botanicus certainly could not have been the success that it has become. Thanks are also in order to the Mellon Foundation and the Keck Foundation without whose financial backing Botanicus would not even have been possible. And last but not least, we would also like to thank you the user for continuing to be active participants in the project (whether you realize it or not). The fact that you use the site and take advantage of the knowledge that is presented makes the entire project worthwhile. Without your support and feedback, there would be no purpose in the existence of Botanicus , and you have proven how important a project of this nature is to the world!

Next stop: two million pages!

Mike Blomberg
Imaging Lab Coordinator
Missouri Botanical Garden

The Temple of Flora

2008-09-30T17:04:00.004-05:00

Published in 1799, Dr. Robert John Thorton’s masterpiece The Temple of Flora remains an awe inspiring work of botanical and artistic literature even in this modern day and age. After inheriting great wealth from his family and inspired by the work of Carl Linnaeus, Thorton set out to produce England’s finest piece of botanical literature to date. Hiring some of the most skilled artists available at the time, Thorton began embarking on this endeavor, however, after some time he unfortunately he ran into endless problems in getting the work published. Due to social, political, and financial factors, he was forced to trim The Temple of Flora down to just a fraction of his original vision for the project. His determination to have this work published left Thorton completely broke at the time of his death, however, his legacy continues to live on through the exquisite The Temple of Flora. Filled with lavishly beautiful full color illustrations, botanical information, and even nods to classic poetry, The Temple of Flora is truly without a doubt one of England’s premier pieces of botanical literature.

Thanks to the publishing company Taschen Books, this extremely rare work is now available to the general public as a set of high quality large format prints. Packaged in a presentation case, the 110-page offering features a 44-page booklet including author Werner Dressendörfer’s introduction as well as the texts of all 31 botanical plates and 33 loose-leaf Elephant folio-sized color prints for browsing or framing. For a preview of Taschen’s publication of these plates, follow this link:
The Temple of Flora @ taschen.com.

The Temple of Flora is also available through Amazon.com at the following location:
The Temple of Flora @ amazon.com

Mike Blomberg
Imaging Lab Coordinator
Missouri Botanical Garden

An attempt at reprinting scanned material

2008-09-25T16:14:00.004-05:00

Recently, we decided to try to reprint a book that we had scanned. We had been interested in trying out the services of Lulu, an internet website that allows users to self-publish their own works by uploading content and having it professionally printed out in quantities as low as just one. The book we ultimately decided upon was Culpeper's English physician and complete herbal. It is a popular title with both text and some interesting line drawing illustrations. We were hoping to try a book with nice full-color images to see how they would translate to a printed page from a modern day 4-color press, but in the end, we still decided that this would be the best book to give printing with Lulu a test run with.

There were some seemingly minor technical issues that we had to deal with before we could actually send the book off to be pressed. In an idea world, it would be great to just send a PDF file of the book as-is and not have to worry about anything. The reality of the situation, however, is that Lulu only offers a certain number of sizes, and we were going to have to resize all of our images in order to meet the criteria for one of these sizes.

Resizing all of the images seemed like a simple enough task. The pages seemed to be about 6.5” x 9.5” (16.5 cm x 24.1 cm) in size, and the closest size that Lulu offers is 6” x 9” (15.2 cm x 22.9 cm). Logically, it made sense to just shrink every page slightly to fit within these dimensions. I created an action using Adobe Photoshop that would automate this process. After running this automated action, I then created a new PDF of the book that would be uploaded to Lulu’s server and be ready for printing.

Then the problems began… I still am not sure exactly what the issue was, but I had a terrible time trying to upload this file to Lulu’s server. The PDF file was about 350MB in size, but for over two weeks, I could not get it to successfully upload. I would check on the progress of the upload from time to time, and I noticed it would always start over. Sometimes it would upload a small portion of the file and then stop, which in turn left numerous incomplete copies in our Lulu account. I was not sure if the problem was with Lulu’s server or perhaps our own Internet service, so I even tried to upload the file from home, but I still could not get it to work. Finally after a couple weeks of frustration (and determination that eventually it would upload successfully), I spoke with a representative at Lulu who also could not figure out what was going on. She suggested that I try it once more and see if it works, and if it did not, we would proceed from there. By this point I didn’t have much faith in their server. I still had quite a few of the incomplete attempts present in the available files for our account. I decided to try once more from scratch and deleted everything in the process before attempting to upload again. Afterwards I uploaded the file from home without any problems. Success!

We were excited to receive the book in the mail about a week later, especially after all the technical issues we had in getting the PDF for it uploaded. Just looking at the book, one can see that Lulu does excellent work. We chose a paperback binding, and the book looked very professional. Flipping through, however, we noticed some problems… Many times text would run into the gutters. In other instances (particularly towards the end of the book), text would be cut off at the top or the side. After digging a little deeper and figuring out what caused this, I discovered that the Photoshop action I used was inherently flawed. Because of the way we scan the books, the cropping on pages as we’re scanning them can be a bit inconsistent. Using the Photoshop action I created, this inconsistency in the size of the images ultimately led to the text getting cropped when printed. It was a very important lesson to learn should we decide to do something along these lines again in the future…

All in all, we were satisfied with the quality of Lulu’s printing and the turnaround time. My only complaint that I really had with Lulu was in the difficulty of uploading the PDF file to their server. Everything else that we were unsatisfied with (text running into the gutter, text being cut off at the edge of pages, etc.) were due to our own technical errors in preparing the material. I have never had to work on material in a prepress manner like this, and in the end, I found this to be a very interesting learning experience!

Mike Blomberg

Imaging Lab Coordinator

Missouri Botanical Garden

New Changes in the Imaging Lab

2008-04-18T15:53:00.004-05:00

It's been awhile since our last blog entry, so we thought we would update everyone on our daily scanning operations...

Exciting stuff has been going on in the imaging deparment! We recently hired three new imaging technicians bringing our total to six, which is the most we've ever had. (Previously, the most we ever have had was five...)

In other news, we have put our Better Light scanning station back into full-time production. This was a major station used in the scanning of large rare books for the Illustrated Garden website. It is a large copy stand setup comprised of a Cambo large format camera and a Better Light scanning back and is surrounded by four hot lights. Once we shifted our focus to Botanicus, this station had unfortunately fallen more and more out of use in favor of the rapid scanning we were achieving with the Indus book scanners. The Better Light produces far better quality scans than the Indus scanners, however, its major drawback its speed. (Since it is a scanning back the imaging sensor moves across the back of the camera much like how a flatbed scanner operates, which in turn creates long scan times as opposed to the instant capture of a CCD in a digital camera.) The good news is that we upgraded to a newer model of Better Light scanning back which is much faster both in scanning speed and data transfer speed, so using the Better Light scanning station as a full-time production station for Botanicus station is much more feasible than before! This station will be useful in adding what we consider "oversized" material (books that are too large to be scanned on the Indus book scanners), so expect to see an increase in the number of folios that are added to Botanicus in the future...

We have also been taking steps lately to further improve our image quality. Every scanning station has undergone calibration methods so that our image captures can be as close to the original as possible. The major drawback to calibrated systems is that the monitors of the end users of Botanicus are not calibrated, so the difference to the average user may not even be noticeable. However, for archiving purposes, it is reaffirming to know that our stored images are more accurate to the source material than ever before in the past.

Mike Blomberg
Imaging Lab Coordinator
Missouri Botanical Garden

Discovered Bibliography by Scientific Name

2007-09-14T09:53:00.001-05:00

We've released new functionality in Botanicus to allow users to search across all the scientific names we've indexed throughout our digital library and view a bibliography of occurrences - what we're tentatively calling a "discovered bibliography". To view in action, begin here:

http://www.botanicus.org/NameSearch.aspx

You can search by any taxonomic name, such as Poa annua, or Poaceae, to return results.

Next steps will be to allow users to search for a taxonomic name & return results for it and its synonyms, or taxa below. But before implementing that advanced functionality, we'd like to make sure this works well for a single taxonomic name. Please give it a try and leave comments below. --Chris Freeland

Index Pages - To tag or not to tag?

2007-08-08T08:57:00.000-05:00

Should the following pages be tagged as PageType "Index"? Do we need other options?

More JP2 tests

2007-07-02T11:32:00.000-05:00

Following last week's tests I've encoded a representative set of images at various JP2 compressions. Download Excel spreadsheet showing file sizes & URLs to images.

JP2 Samples for review

2007-06-29T15:15:00.000-05:00

In our current workflow, we convert TIFs from our scanner(s) into 100% lossless JP2 using LuraTech's SDK. We're evaluating lower quality JP2, and your comments are appreciated for the following images. This sample was taken from a volume of Curtis' Botanical Magazine. There are 211 images in the volume.

Total file size for all uncompressed TIFFs: 4.39GB
Total file size for 100% JP2: 2.28GB
Total file size for 90% JP2: 660MB
Total file size for 80% JP2: 499MB
Total file size for 50% JP2: 160MB
Total file size for 30% JP2: 58.5MB

Illustration - TIFF 21.6MB

100% quality JP2
Size: 12.6MB

90%
Size: 2.8MB

80%
Size: 2.0MB

50%
Size: 527KB

30%
Size: 184KB

Text - TIFF 21.0MB

100% quality JP2
Size: 10.7MB

90%
Size: 2.8MB

80%
Size: 2.0MB

50%
Size: 565KB

30%
Size: 208KB

Happy Birthday Carl!

2007-05-23T10:02:00.000-05:00

Today is the 300th birthday of Carolus Linnaeus, often called the "father of modern taxonomy" because of his revolutionary system for naming organisms. View a list of digitized titles by Linnaeus in Botanicus, or use the links below:

Caroli a Linne´ Species plantarum :exhibentes plantas rite cognitas, ad genera relatas, cum differentiis specificis, nominibus trivialibus, synonymis selectis, locis natalibus, secundum systema sexuale digestas.
Publication info: Berolini :Impensis G.C. Nauk,1797-1830.

Caroli Linnaei ... flora Suecica :exhibens plantas per regnum Sveciae crescentes, systematice cum differentiis specierum, synonymis autorum, nominibus incolarum, solo locorum, usu pharmacopaeorum.
Publication info: Stockholmiae :Sumtu & literis Laurentii Salvii,1745.

Caroli Linnaei ... hortus Upsaliensis :exhibens plantas exoticas, Horto upsaliensis academiae a sese illatas, ab anno 1742, in annum 1748, additis differentiis, synonymis, habitationibus, hospitiis, rariorumque descriptionibus, in gratiam studiosae juventutis.
Publication info: Stockholmiae : Sumtu & literis Laurentii Salvii, 1748.

Caroli Linnaei ... Species plantarum :exhibentes plantas rite cognitas, ad genera relatas, cum differentiis specificis, nominibus trivialibus, synonymis selectis, locis natalibus, secundum systema sexuale digestas...
Publication info: Holmiae : Impensis Laurentii Salvii, 1753.

Caroli Linnaei, Sac. Reg. Maj. Suec. ... Amoenitates academicae, seu, Dissertationes variae :physicae, medicae, botanicae antehac seorsim editae nunc collectae et auctae cum tabulis aeneis, accedit Hypothesis nova de febrium intermittentium causa.
Publication info: Lugduni Batavorum :Apud Cornelium Haak,1749.

Caroli Linnaei...Systema naturae per regna tria naturae :secundum classes, ordines, genera, species, cum characteribus, differentiis, synonymis, locis.
Publication info: Holmiae : Impensis Direct. Laurentii Salvii, 1758-1759.

Caroli v. Linne´ Equ. aur. de stella polari ... Genera plantarum :eorumque characteres naturales secundum numerum, figuram, situm, et proportionem omnium fructificationis partium.
Publication info: Holmiae :Impensis direct. Laurentii Salvii,1764.

Genera plantarum :eorumque characteres naturales secundum numerum, figuram, situm, et proportionem omnium fructificationis partium /by Caroli Linnæi.
Publication info: Holmiæ (Stockholm) :Impensis Laurentii Salvii ,1754.

Hortus Cliffortianus :plantas exhibens quas in hortis tam vivis quam siccis, Hartecampi in Hollandia, coluit ... Georgius Clifford ... reductis varietatibus ad species, speciebus ad genera, generibus ad classes, adjectis locis plantarum natalibus differentiisque specierum. Cum tabulis Ã¦neis /auctore Carolo LinnÃ¦o.
Publication info: AmstelÃ¦dami : [s.n.], 1737.

Supplementum plantarum Systematis vegetabilium editionis decimae tertiae, Generum plantarum editionis sextae, et Specierum plantarum editionis secunda /editum a Carolo a LinneÌ ...
Publication info: Brunsvigae : Impensis Orphanotrophei, 1781.

Systema naturae, per regna tria naturae :secundum classes, ordines, genera, species cum characteribus, differentiis, synonymis, locis /Caroli a Linne´ ...
Publication info: Vindobonae [Vienna] :Typis Ioannis Thomae ...,1767-1770.

Browsing by Tag Cloud & Google Maps now available

2007-04-23T09:52:00.000-05:00

We've enhanced our Browse functionality to now include browsing by:

1. Library of Congress Subject Heading Tag Cloud
http://www.botanicus.org/Default.aspx?browseType=cloud
When we parse MARCXML into Botanicus, we extract 6xx & 7xx into locally-defined subject fields in the Botanicus database. We then display those subjects in a tag cloud, as shown by the link. Clicking a tag will take you to a list of all titles with that tag.

2. Google Maps
http://www.botanicus.org/Default.aspx?browseType=map
We parse out locations in 6xx z$ and geocode them using the Google Maps API. We then map those geocoded locations and provide a list of the titles tagged with that location.

Sheet Feed Scanning

2007-03-12T16:38:00.001-05:00

From Mike Blomberg, Senior Imaging Technician:

I have devoted the past two months and then some towards scanning a majority of volumes of the Annals Of The Missouri Botanical Garden (excluding issues from the past two years). I thought it would be interesting to shed some insight as to production work with the Kodak i280 sheet feed scanner as opposed to the Indus 5002 scanners that we use for most of our scanning work.

All in all 92 volumes of the Annals will be captured. Even though this is certainly not the longest run we've worked on (take Flora for example), it's good to be wrapping up work on it finally. Out of those 92 volumes, only 22 of them were bound copies scanned with the Indus scanners. (This was only because we didn't have extra copies of those particular volumes that we could sacrifice to the guillotine.)

On average, it takes about an hour to scan a typical book using the Kodak sheet feed scanner. If you assume that about five volumes could be completed during a normal work day, I only estimated that scanning the 70 volumes of the Annals designated for sheet feed scanning would only take about a month to scan. Unfortunately, the Annals took much longer than originally anticipated.

Besides the sheet feed station being down for two different periods where it was used to evaulate two different microfiche scanners, other factors came into play that pushed the completion of scanning back much further than expected. One example is that the volumes started getting longer and longer as the run progressed. By the 1960's, it wasn't uncommon for volumes to be twice the size that the early Annals volumes were. Some volumes had page counts upwards of 1200-1400 pages as opposed to the average page count of around 700-800 pages.

This run has been the most consistent that the i280 has been used by us. Normally it would only be used for a handful of volumes in a given run that were deemed suitable for sheet feed scanning (books that were already falling apart and such). One major issue that needs to be attended to with the i280 is dirt accumulation... With this much usage of the machine, cleaning needed to be done much more frequently (usually once or two a volume). Dirt easily gathers on the various components of the machine given the nature of paper. The two biggest culprits in slowing down production are dirt on the scanning heads and dirt on the feed rollers. Dirt on the scanning heads obviously create problems with image quality, however when dirt collects on the rollers, it becomes difficult for the machine to feed pages into the machine.

One of the most interesting tests that this run presented was to see how the Kodak i280 handles different types of paper. Up until this point, everything that has been run through the machine has been very dull matte paper. However, sometime in the 1960's, the Annals were starting to be printed on glossy paper instead. I didn't expect much of a difference in the performance of the machine, but that proved to not be case. The glossy pages tended to stick together more often resulting in what the Kodak software calls "multifeeds" (where more than one sheet is accidentally fed through the scanner). The scanner can be set to stop when it encounters a multifeed, however, with the glossy pages, a number of multifeeds went through undetected by the scanner. The glossy paper of these volumes also tended to put a strange film on the feed rollers that would have to be regularly cleaned off (usually a few times in each volume). It was different from typical dirt and dust that gather on the rollers, but the effects were the same if not worse! I doubt we'll be scanning many books with glossy pages considering the focus of our scanning is for pre-1925 literature, but at the very least it was interesting to see how the i280 reacted differently to glossy paper.

Sheet feed scanning of the Annals of the Missouri Botanical Garden is completed as of this point. Right now, just a few gaps are being filled in with bound volumes being scanned on the Indus 5002 machines. Next up to be sheet feed scanned: Rhodora. It is comprised of 70 volumes and every single volume is to be scanned using the i280... Luckily these volumes are smaller both in page count and in physical size so the scanning should proceed much quicker than with the Annals.

I hope this entry sheds some light into the trials and tribulations of sheet feed scanning!

Mike Blomberg, Senior Imaging Technician
Missouri Botanical Garden

Better citation resolving and other web site updates

2007-03-12T16:38:00.000-05:00

We've made the following updates & changes to the Botanicus web site, including better integration with Tropicos & more helpful screens resolving protologue citations. You can view these changes at http://www.botanicus.org

• Tropicos names integration.
• Tropicos and uBio names displayed in the same panel.
• Search text box moved from left hand side to the header above the tabs.
• Increased height of title browsing tabs in the header.
• Updated logic for check for existence of PDF files.
• Updated logic for display of OCR text.
• Updates to the display of title and item drop down lists in IE 6 and below.
• Added better exception handling to display generic error and page not found pages.
• New page resolve logic from Tropicos that only displays a citation if an exact match was found. Similar results displayed if no exact match could be found. To view this in action, follow the "View in Botanicus" link at http://mobot.mobot.org/cgi-bin/search_vast?onda=N24201228

Unfortunately we're still chasing down a bug with some browsers over how they cache scripts for Botanicus. Hit your browser's Refresh button if areas of the web site appear to overlap or are not evenly aligned.

LCSH, geotags & Google Maps mashup

2007-01-19T13:07:00.001-06:00

We took strings from our MARC 650 $z and plotted the corresponding titles using CommunityWalk.com, a Google Maps-based API.

14 hours in the life of Curtis's Botanical Magazine v.27-28

2006-12-19T17:07:00.000-06:00

We are now caught up with the backlog of volumes waiting to be published, so we're getting accurate statistics for timing, rate of scanning, rate of publishing, etc. Here's a representative timeline for Curtis v.27-28, originally published in 1808, scanned 12/18/06:

December 18

1:20PM - Imaging Technician scans first page of volume on Indus 5002
4:27PM - Imaging Tech scans last page (#255)
8:00PM - PageConvert runs on Indus 5002 & begins creating JP2s
8:21PM - PageConvert finishes creating JP2s & begins copying TIFs & JP2s to SAN
8:45PM - PageConvert finishes copying files and sets Item's Status to 30 ("on server, but not published") in database

December 19

2:00AM - PagePublish runs, adds page-level metadata to Botanicus, sets Item's Status to 40 ("on server & publish ready") & outputs OCR job file

Book is now "online" and visible in Botanicus

2:03AM - PrimeOCR picks up job file & begins text conversion

3:03AM - PrimeOCR finishes text conversion

So, we went from pulling a (relatively small) book off the shelf to online w/OCR in 14 hours, 8.5hrs of which was downtime between scheduled processes.

New items added daily, complete with RSS

2006-11-16T16:06:00.000-06:00

We're back in production after a 2 month hiatus from publishing newly digitized volumes. We had several behind-the-scenes tasks to do (SAN rebuild & restore, SQL 2000 to 2005 migration, web updates) and now that those have been successfully completed we're regularly publishing new volumes. You can view new items at:

http://www.botanicus.org/recent.asp

Or, subscribe to our RSS feed at:

http://www.botanicus.org/rss.xml

Now serving from SQL Server 2005

2006-10-12T14:02:00.000-05:00

Botanicus.org is now being fed bibliographic & page-level metadata from SQL Server 2005 & a 64-bit AMD Opteron. Page generation & loading are considerably faster.

Browse publication dates, titles, and authors

2006-09-13T17:22:00.000-05:00

We've implemented functionality that allows users to browse digitized titles by title, author, and publication start date. Give it a try at:

http://www.botanicus.org/browse.asp

New search functionality implemented

2006-08-29T12:54:00.000-05:00

We've implemented a new search feature that allows users to search for a term within titles, authors & text. Right now this is a simple search (no boolean or proximity) that will locate your exact search term; advanced functionality is forthcoming. Please give it a try at:

http://www.botanicus.org/search.asp

As always, we welcome all feedback! Please send directly to "chris dot freeland at mobot dot org"

PDF comparison with LuraDocument

2006-08-24T14:08:00.000-05:00

As part of our current scanning procedures, our Imaging Techs take our full resolution TIF images and convert them to bitonal GIFs using a Photoshop action. They then take those GIFs and pull them into a PDF so that users can download an entire volume for offline viewing - "grab and go" as I like to call it. We've been downsampling to a bitonal GIF so that the PDF is relatively small, but even then they are usually between 30-70MB. Not to mention they're bitonal, which is fine for text but terrible for illustrations!

Download a PDF made from bitonal GIFs (26MB)

So imagine my delight after evaluating LuraTech's LuraDocument and being able to compress full resolution (and color depth) images down to an 11MB PDF:

Download this better PDF made from color & greyscale images (11MB)
*Note - we scan text in greyscale and color illustrations in color. This was done to save server space, but we're reconsidering this approach and will probably scan everything in color.

**Note - There are watermarks on every page because this is an evaluation version of the software.

Users are harvesting our PDFs en masse, especially those in remote corners of the world without ready access to the internet, so we'd like to provide the best possible PDF. Our intention is still to provide full quality images via our web-based page navigation interface; this is just a better option for those who want to grab the entire PDF in one fell download.

List of authors with digitized titles

2006-08-21T16:58:00.000-05:00

One of the popular features of our IllustratedGarden web site was a single-page view of all of our authors with digitized works, making it easy for users to browse and see if a particular author was represented. That functionality has been implemented in Botanicus:

View Authors in Botanicus

JPEG2000 images online for select titles

2006-08-10T16:15:00.001-05:00

We have evaluated LuraTech's LuraWave SDK for JPEG2000 (JP2) encoding to great success. On average it takes 20-30 minutes to encode all of the TIF images for a digitized book into JP2 images, which is about 3x faster than our OCR conversion rate (apples to oranges, I know, but lets you know that it takes about 2 hours total to post process a book after it's been scanned). We still have lots of encoding parameters to evaluate before moving into a production environment, but until then, please try the zoom & pan functionality for the following titles:

Caroli Linnaei ... Species plantarum

Synopsis plantarum,seu Enchiridium botanicum, complectens enumerationem systematicam specierum hucusque cognitarum /curante C.H. Persoon.

Joannis Hedwig...species muscorum frondosorum descriptae et tabulis aeneis lxxvii coloratis illustratae /opus posthumum, editum a Friderico Schwaegrichen.

View this page to see how the JP2 images look for color illustrations.

Creator links and rare books

2006-07-03T12:03:00.000-05:00

Botanicus.org is actually an expansion of one of our most successful digitization projects, Rare Books from the MBG Library. The selection criteria for that project was a bit different than our current method, with an emphasis on beautifully illustrated, rare titles. One aspect of that project that many visitors enjoyed were descriptions of significant titles, written to put the title in historical context. We've now migrated all of those titles into Botanicus and included the descriptions. View Köhler's Medizinal-Pflanzen, our most popular title.

Another new addition to Botanicus is a page that lists all of the titles we've digitized for a given author, plus a biography (if present). Try it out for Nikolaus Joseph Jacquin.

A different view on prioritized titles

2006-06-26T17:12:00.000-05:00

Doug Holland, Director of MBG Library, threw out an interesting challenge - reevaluate our prioritized list to include the number of volumes for a title, and rank the titles by a ratio of number of names to number of volumes. We wanted to do this at the beginning of the project, but there was no way of knowing how many volumes we had for a given title (or any easy way to pull it from Millenium, our library software). Since we've exported data out of Millenium and into Access, that's now a simple task.

View list

One caveat - the number of volumes for a title is for the entire run as cataloged in our library. So, there are more volumes of say, Curtis's Botanical Magazine, in the catalog than we plan to scan because we're going to stop at 1923. Unfortunately there's no definitive way to pull out just the volumes dated earlier than 1923.

One of the more interesting aspects of looking at our prioritized literature in this way is that if we started in on this list, our number of digitized titles will grow quickly since many of these are single volume works. Right now we're plateau'ing on titles because we're just adding in more volumes for the titles we've already begun. In practice we plan to pull from both lists (this one and our previous prioritized list).

Integrating taxonomic intelligence from uBio

2006-06-09T16:54:00.000-05:00

David Remsen and his team of developers working on uBio have created some amazing web services for taxonomic intelligence. We've integrated LinkIT with the OCR'd text from Botanicus. See it in action at:

http://www.botanicus.org/text.asp?pageid=93935

Click the "View scientific names on this page" link to call LinkIT and discover the names within the text.

JPEG2000 interface to texts and illustrations

2006-05-31T17:29:00.001-05:00

We have used LizardTech's MrSID image format since 2000 to display our herbarium specimen images. It's worked very well for us over the years, but we were always concerned that it was a proprietary format. The latest version of LizardTech's ExpressServer now supports JPEG2000 (*.jp2) images, which are based on an open standard, so we began looking for ways to create an Ajaxian interface on top of ExpressServer. Luckily we found previous work in the form of the original GSV from Michal Migurski and GSIV from David Allen. Jay Paige, one of MBG's intrepid programmers, customized the core app to work with ExpressServer, and voila:

View example text page

View example illustration

View specimen

We're finalizing credits and such on the code and will make it available to anyone who wants it.

Rudimentary page turning for 399 volumes

2006-05-19T19:37:00.000-05:00

Today I published a rudimentary page turning interface for the 399 volumes we've scanned to date. That's 232,000+ pages from 23 titles, and represents over a year's worth of digitization. The interface is still VERY basic and does not match the interface I'm proposing. My main reason for doing this was to test our data model in a web environment and it's already paid off - we discovered some missing relationships that we need to incorporate into the next version of the model.

A note about the images: We are scanning materials at 400dpi grayscale with our Indus book scanners and saving the master images as uncompressed TIFs. The images presented on the site are bitonal GIFs and will be replaced with multi-resolutional, grayscale JP2 (JPEG2000) images once we settle on a vendor for encoding.

Please have a look at http://www.botanicus.org/tobescanned.asp and provide feedback!

OCR results for Species Plantarum

2006-05-12T10:38:00.000-05:00

One of the titles we've digitized is Linnaeus' Species Plantarum, published in 1753. It's arguably the most significant title in plant taxonomy, and our copy posed some challenges because it's tightly bound and has a fair amount of bleed-through on the page. You can download a representative page (4MB TIF or 1MB JPG) to see what I mean. This page in particular is important because it's where the scientific name for corn, Zea mays, was first published.

I expected the OCR to be bad, but was shocked at how miserable it was with our standard settings! View the text to see what I mean. Terrible, eh?!

So, I tried to make it better. I used Prime's internal image cleaning routines (deskew, despeckle, noise reduction) to see if there was improvement. There was, but it still wasn't enough. Check out the file.

Finally, I went back to the original TIF (a copy, actually) and using Photoshop changed the Threshold to approximate a good bitonal image. Here's a lower-res GIF to view as an example (OCR was on the full-res TIF). I had heard antecdotally that bitonal images resulted in more accurate OCR. I would have to agree at least for this one page - view the results to see the dramatic improvement. Far from perfect, but a heck of a lot better than the earlier tests!

Martin Kalfatovic at Smithsonian kindly offered to run the TIF through LuraTech's software, which uses ABBY as the OCR engine. The results on the unedited TIF were actually pretty good (certainly much better than the results from Prime). View the file.

What this says to me is that we should evaluate the standard OCR results, and if they're bad (how to judge?), maybe make derivatives just for the purpose of OCR. Or, switch to ABBY!

Searching the text of 300 digitized titles

2006-05-12T10:20:00.000-05:00

We're running PrimeRecognition's PrimeOCR for text conversion. I've spent several months working out kinks and learning about how OCR really works (or doesn't work) with our historic literature. We have a cache of 83,000 text pages generated from Prime and I wondered how easy it would be to drop in existing services within our network to start interacting with this text. Turns out it was REALLY easy. We're a Windows/.NET shop, so we have several machines running IIS 6.0. I built an out-of-the-box Indexing Service implementation and incorporated it into our beta site at:

http://www.botanicus.org/search.asp

Give it a try - results are interesting!

Prototype interface for scientific literature

2006-05-12T10:08:00.000-05:00

I had an "Aha!" moment about 2 weeks ago while reading Pragmatic Ajax by Justin Gehtland, Ben Galbraith, and Dion Almaer (highly recommended, by the way). The authors use Google Maps as a classic example of Ajaxian techniques and it struck me that the interface for Google Maps is almost identical to what we've been envisioning for zooming & panning digitized scientific literature with JPEG2000. One idea led to another and 12 hours later I had the beginnings of a web interface. I started putting ideas into PowerPoint (one of the easiest tools to use for prototyping) and came up with the following:

Download 4MB PowerPoint (zipped)