Not a Luddite, but a Controlled-Digitization Advocate?

September 29, 2006

The success of mining large scale digital collections – whether they be one million books or many more audio, video and image files seems to depend on two things: the organization of the analog-to-digital conversion of the collection and the robustness of the search engine used for the mining.  Google’s goal of digitizing the entire world library – if it even comes close to succeeding – will end up just as inaccessible as the Library of Babel if it is not done with considerable forethought and not accompanied by a very powerful and adaptable search engine.  This week’s readings about syllabus finders and search techniques provide a good overview of search engine logic, and the syllabus finder obviously works well for fairly narrow search criteria, but I wonder what the product looks like.  The syllabus searcher still has to sort through a plethora of “hits” since just about every undergraduate course in every decent university has a website or at least the syllabus online.  What sort of care was put into digitizing the information?  As syllabi are typically posted directly on the site, my guess is that the results of a syllabus search are generally clear.  What about shelves of library books that are simply scanned into a series of incunabula files?
Recently I worked on an ongoing information fusion project in support of Homeland Security.  The goal was to provide response agencies with access to the vast amounts of information that are already being collected regarding marine commercial and recreational traffic – millions of vessels every day.  One of the primary risks to the success of that project is that it will be too successful – that the amount of information available to security agencies will simply overwhelm them and actually hinder their responsiveness.
I’m not a luddite by any stretch, but I am in favor of a controlled evolution from analog to digital.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: