Social Media Archives Toolkit

In 2014, the NC State University Libraries was awarded an EZ Innovation Grant from the State Library of North Carolina to develop a freely available web-based documentary toolkit that publicly documents our own effort to develop a sophisticated social media archival program in a way that may help guide cultural heritage organizations that are interested in collecting and curating social media content.

Much has been written and shared about archiving social media since the development of the toolkit, and the ethical issues of doing has been a large part of the discussion. A significant community in this area is Documenting the Now. DocNow had less of public presence when the toolkit was released. We have not updated the toolkit to include their work (or that of others that has been shared publicly since 2014-2015), and we encourage readers to visit their site.

In order to develop accurate historical assessments, researchers must have access to primary materials that are represent a large and diverse set of participants. Social media platforms have become a venue where serious discourse and creation take place, but much of this critical and ephemeral content is lost to researchers as few institutions are collect and preserve this content. Tools are surfacing (including Lentil, developed by NC State University Libraries) that will enable cultural heritage institutions to preserve the social media record at a much lower cost than was previously possible. While these tools are becoming more sophisticated, there are limited guidelines for developing curatorial policies around these tools.

The toolkit may aid cultural heritage organizations to:

  • Develop collecting strategies for content such as official communication from their institution as well as crowdsourced communication from communities of interest
  • Gain knowledge of how peer institutions have gone about collecting similar content
  • Develop an understanding of the current and potential use of social media content by researchers
  • Assess the legal and ethical implications of archiving social media content for future use by researchers
  • Develop techniques for enriching collections at a minimal cost by taking advantage of harvesting interfaces provided by social media platforms

The goal of the toolkit is to have a meaningful impact on archival researchers by promoting the inclusion in the historical record of a larger and more diverse set of perspectives found through social media platforms.

As part of this grant, the Libraries had also developed the Social Media Combine, a pre-configured collection of tools including Social Feed Manager and Lentil for easily building Twitter and Instagram social media archives on your own computer. However, this application is no longer maintained, in part because Facebook no longer supports the API Lentil relied upon.

The EZ Innovation Grants were made possible through funding from the federal Institute of Museum and Library Services (IMLS) under the provisions of the Library Services and Technology Act (LSTA) as administered by the State Library of North Carolina, a division of the North Carolina Department of Cultural Resources.

This project was a collaboration between the Digital Library Initiatives department, the Special Collections Research Center, and the Copyright and Digital Scholarship Center at NC State University Libraries. We would like to express our gratitude to the students who contributed to this project:

  • Aditya Ambre
  • Jessica Dixon
  • Abigail Jones
  • Aniket Lawande
  • Kahlee Leingang
  • Ramakant Moka
  • Mitul Panchal
  • Jessica Serrao
  • Ashlyn Velte

The NC State University Libraries welcomes feedback on its projects, including this one. Please contact us!