Counting Bytes, Not Boxes: Born-Digital Collection Extents with Python

“How many linear feet do you have?” In special collections and archives, we often hear this question. A linear foot is a unit of measurement used to gauge the size of a collection or an organization’s holdings. As a digital archivist, I work with born-digital materials found on physical media that also occupy physical shelf space. However, it is more meaningful for us to measure these materials by the amount of disk space they consume rather than their physical footprint. My analogous units of measurement are the number of bytes and files. Answering questions such as, “How many gigabytes are in this collection?” or “How many files are in this series?” was time-consuming, so I wrote a Python script that calculates these totals.

When we process and describe born-digital materials, we include the total number of bytes and files as part of the material description. We record these figures for each born-digital item, each series, and the total collection. Although the software we use to manage our collections, ArchivesSpace, has an Extent Calculator, that feature is designed for materials located in physical containers. Consequently, before I wrote my script, we had to calculate the total number of bytes and files manually. If a collection only had a few floppy disks, that was feasible. However, for a larger collection with hundreds of disks, calculating totals could become unwieldy. In addition to easing the workload for born-digital processors, the script also aids in reporting alongside our accessioning, processing, and preservation ingest figures.

To use the script, we download the Encoded Archival Description (EAD) version of a collection guide. This is in XML, a format used for storing and sharing structured data. The script uses Beautiful Soup, a Python library—reusable pieces of code—that parses XML files. Specifically, the script parses the <c> tags, which represent different components of a collection. The flexibility of EAD posed some challenges. For example, the <c> tag is used for series, subseries, and file records. While there is a <c> tag attribute for the level of description, our EAD doesn’t use this. Getting the script to determine if a component was a series or a file record outside of a series was tricky. 

I originally wrote the script in 2022, and since then, I’ve made improvements and fixed bugs in response to our graduate student assistants’ feedback. Because this is one of the longest Python scripts I’ve written, I’m sure I’ll continue to find ways to improve it over time. With the upcoming release of EAD version 4, I may also have to revise the script, as it was designed to work with EAD3.

The script can be run over a single or multiple EAD XML files. While we’ve tested and mostly used the script in a Docker image, it could be used in any Python environment. View the code repository to download and learn how to run the script.

The D. H. Hill Jr. Library will be closed beginning on Monday, May 5, 2025, for planned electrical work. The Special Collections Research Center is located in Hill Library, and will be closed for all appointments during this period. Special Collections can only make appointment requests after August 3, 2025. During the closure, Special Collections staff will be available to answer questions about our collections, if they are accessible, and to support researchers using digital materials for their research. Some materials will not be available for reproduction requests. Please contact us!