Interoperability Tools

From NEH Project
Jump to: navigation, search


About the Interoperability Tools

The Interoperability Tools were produced after thorough project research into existing exchange technologies. BagIt was early on selected as the most flexible starter technology with the understanding that its adaptability and scalability would be put to the test in the project's experiments with packaging and exchanging diverse types and sizes of digital newspaper collections between content holders and three representative distributed digital preservation (DDP) systems - namely MetaArchive Cooperative, Chronopolis, and University of North Texas Libraries's Coda Repository.

The following scripts and utilities were developed as a result of these experiments. They work in conjunction with the BagIt model. They facilitated the successful packaging and exchange of both digitized and born-digital newspaper collections of all sizes and organization schemes. The scripts and utilities are generalizable enough to be used as is (or in slightly modified form) in numerous settings and content use cases.

Interoperability Tools

The Interoperability Tools are described below and available on GitHub at: For more information feel free to contact

Bag Split/Unsplit Utility

A tool to help with splitting and unsplitting large bags of digital content. This dual-purpose script can decompose large Bags of data into smaller, validated sub-Bag units to ensure more efficient network transfer and on-going archival management (e.g., fixity processing). The script can be repurposed to recompile the smaller sub-Bag units into the original large Bag and validate the results for recovery purposes.

LOCKSSsum Validator

This is a web service that has been produced for LOCKSS-based networks to compare a BagIt manifest with a list of corresponding checksums produced by a LOCKSS box for the purposes of ongoing audit. It is very lightweight and could be easily modified to be of use in non-LOCKSS based networks.

Bad Filename Detection Tool

This script can detect problematic system files (e.g., .DS_Store or Thumbs.db) or poorly formatted files (e.g., filenames with spaces or bad characters) that interfere with the transfer and validation of BagIt packages.

Bag Description Service

This script automates the application of the FCLA Description Service to iterate over top of a set of BagIt-based content. It creates and stores PREMIS records for that data, and regenerates the Bag files. These steps result in a set of per-file technical metadata that now accompany the content as it moves forward for preservation.

Personal tools