Using AI to Help Write a Collection Guide: A Test Case

Written by Grace Midha with contributions from Cathy Dorin-Black

Here at the Special Collections Research Center, we consistently receive new archival materials to add to our collections, which require an online collection guide where researchers can browse descriptions of the materials. Collection guides identify, describe, and list the contents of collections of materials that are not individually cataloged. Most guides also contain information about the person or organization that created the collection. We have wondered how using AI tools could assist in our work in creating these guides. Therefore, we recently decided to experiment with using ChatGPT to write parts of a collection guide for one of our incoming collections, the William P. Rose Drawings of I. O. Jones House. This collection contains eleven blueprint drawings of a Queen Anne style house designed by architect William P. Rose in 1903. 

Why use AI?

Each collection guide includes a biographical or historical note that describes the creator (in this case, William P. Rose) and subject(s) of the collection (the I. O. Jones House). When I began writing the collection guide, I found one website detailing the history of the I. O. Jones House to reference for writing the note. I determined this website was reliable because it is the official website for the Town of Wake Forest, where the house is located. The blueprints themselves contained no supporting historical documentation about the architect, so we knew we would summarize information from the website to write the collection guide. It made sense to use this collection to test how we might be able to incorporate the use of AI into our daily practices, particularly in summarizing information already available on the web.

The Process

For this experiment, we used ChatGPT, a conversational AI tool that helps generate human-like text. In constructing the prompt, I made sure to be specific and provide important context for the model to better understand what I was asking and decrease the chance of hallucinations, as directed by “Prompt Engineering Best Practices for ChatGPT “. I began by explaining that I was writing a collection guide (or finding aid) for a manuscript collection. I described how the biographical/historical note is usually structured: one to two paragraphs that include background information on the subject and creator of the collection, as well as its potential research uses. I also provided historical information about the I. O. Jones house from the website I had found and asked ChatGPT to incorporate any relevant information from the description. 

The first paragraph the ChatGPT returned was fairly well-written and specific, including descriptive information about the house itself. The second, about William P. Rose, was vague, and some of the information was suspect and overly laudatory (referring to Rose as a “trained architectural illustrator known for meticulous renderings”). When I asked ChatGPT where it got that information, it explained that the paragraph on Rose was a “placeholder”; it was based on the common format and tone of collection guides written about architects, and the information had not been obtained from a verified source because none was provided by the prompt. After finding credible biographical information on William P. Rose, I was able to paste it into ChatGPT, asking it to rewrite the paragraph on Rose. GPT was then able to produce a paragraph that was much more specific, including actual information on Rose’s life.

 I made several revisions to the notes produced by ChatGPT before it was published online. I clarified the language, like changing the phrase “circa 1903” to “in 1903.” I removed subjective phrases such as “hallmark elements.” I also revised “self-taught architect and prolific builder” to the more straightforward “architect and builder.” I added more specific information, changing the sentence “one of the region’s leading firms” to “one of Eastern and Central North Carolina’s leading contracting firms.” I also incorporated biographical details about Rose’s life, including the fact that he owned a building supply business and a farm in his later life.  Here is the finished note as it appears in the collection guide:

Citing ChatGPT

Determining how to cite ChatGPT proved to be a bit of a challenge. We considered: 

“William P. Rose Drawings” prompt. ChatGPT, 5 May, GPT-4o, OpenAI, 6 June 2025, chat.openai.com/chat.

This did not seem sufficient, since it lacked the websites that we provided to ChatGPT. We then consulted several sources, such as the American Library Association (ALA), MLA (Modern Language Association, and the Society of American Archivists (SAA), and didn’t find a lot of guidance. MLA, however, provided examples that included listing the prompt that was given to ChatGPT. So we tried again in this format:

Prompt: “I work at an archive and am writing a finding aid for a collection that we acquired. The collection is titled ‘William P. Rose Drawings of I. O. Jones House.’ Usually, finding aids are about 1-2 paragraphs long. Given this information about the house: [text pasted from https://www.wakeforestnc.gov/planning/long-range-planning/historic-preservation/local-landmarks] and this information about the architect [text pasted from https://ncarchitects.lib.ncsu.edu/people/P000132], write a finding aid,” prompt, ChatGPT, 5 May, GPT-4o, OpenAI, 6 June 2025, chat.openai.com/chat.

This citation provided plenty of context but seemed too long. We decided on a condensed version of that, which reads: 

 “Written with assistance from ChatGPT-4o, OpenAI, which was used to summarize content found on these pages: Town of Wake Forest Local Landmarks and NC Architects and Builders William P. Rose. Text was revised for accuracy and clarity.”

We felt this conveyed the necessary information (including the cited websites) without being too long. 

Conclusion

We conducted this test to find out if AI could be useful in helping write collection guides, only in cases where we would be condensing trustworthy information already available on the web.  Typically, the best way to glean information about creators is from the collections themselves, in the form of correspondence, professional files, diaries, etc. AI cannot replace learning from these primary sources. In this particular case, we only had drawings and nothing else to draw from, so it seemed like a good test case for using AI.

Lessons learned:

• The best results come from the most precise prompts.
• Even with a good prompt, AI tools will potentially hallucinate or give inaccurate information. Double-check results and ask where it got the information!
• ChatGPT will include exaggerated, flowery, or subjective language that we do not include in our guides.
• ChatGPT gave us more information than we needed when we would rather keep the notes brief.
• Citing the use of AI does not seem to be standardized yet, but it is crucial to be as transparent as possible about using it, as well as any websites that are provided to it.

It is difficult to say if using this method will actually save time in the long run. Since this was our first test, it definitely took longer than writing the guide the “old-fashioned way,” but that does not mean it wouldn’t be a useful tool going forward now that we have elements of a workflow in place. If we do continue to use AI to construct historical notes, we will continue to approach it with healthy skepticism.

Learn more about the Special Collection Research Center, or visit our Rare and Unique Digital Collections for access to more than 1.5 million images, videos, audio recordings, and textual materials. You can also visit Historical State for a timeline of the history of NC State, and explore our collection guides.

If you have questions about our collections or would like to view Special Collections materials in person, please contact us at library_specialcollections@ncsu.edu or submit a request online. Using our collections is by appointment only. Appointments are available Monday-Friday, 9am-6pm, and Saturday, 1pm-5pm. Requests for a Saturday appointment must be received no later than Tuesday of the same week.