Moving the September 11 Digital Archive to Omeka

It’s hard for me to comprehend that the September 11 Digital Archive has been in operation for over nine years. I was brought on near the beginning of the project, charged by the Center for History and New Media to make sense of the growing amount of contributed material. As demand to preserve reactions to the attacks increased beyond our initial capacity, we began to accept CD-ROMs and Zip drives filled with heterogeneous collections. In addition, as our reputation grew, other 9/11-related projects donated their digital content, often in the form of fully functional websites.

Quickly we realized that our original database was insufficient to the task, so we designed one better suited to store a multitude of files and complex directory trees. Over the years, this database and accompanying website (designed primarily for researchers) have done their job satisfactorily. We donated a copy of the Archive to the Library of Congress, and, as contributions waned, the Center focused on building other websites and digital tools.

The Center’s work on the Archive and other collections websites greatly inspired the development of Omeka, an open source, standards-based web publishing platform designed especially for cultural heritage institutions. Our mission was to provide a generalized CMS that would benefit a wide variety of collections projects, to facilitate the ingestion, description, organization, and presentation of digital content. It’s fitting, then, that we’ve chosen Omeka as the next home of the September 11 Digital Archive.

By far our biggest job is moving a massive amount of content to Omeka. The Archive contains hundreds of collections and well over 100,000 digital objects. Each collection has particular characteristics and many have unique requirements for import. To avoid laborious and redundant work, I’m working on a framework that systematizes the import process. With this framework in place, all I need to do is evaluate the Archive’s holdings, write an import strategy for every collection or class of collections, and run a script. If something goes wrong during import, the script picks up where it last left off.

Data loss during import is unacceptable. Thankfully, mapping the Archive’s data model to Omeka is proving to be straightforward, given Omeka’s database pedigree and overall extensibility. When the need arises, I simply create element sets, item types, and plugins to accommodate data structures and behavior that are not native to Omeka. This is a testament to the success of Omeka as a generalized CMS, able to adapt to virtually any circumstance.

It continues to be a mixed blessing working on the September 11 Digital Archive. The work offers exceptional methodological and technological use cases for working with digital archives, but the material is a constant reminder of the September 11, 2001 attacks and the response to them. We’re certain that the move to Omeka will enrich the historical record by making such noteworthy material even more usable by scholars and accessible to the general public.