Collection Guides (Finding Aids) Usability Study of Novice Users (2010) | User Studies

Overview

This study was designed to evaluate the usability of the Collection Guides Web site (otherwise known as the Finding Aids Web site), designed by the Special Collections Research Center, from the point of view of the novice user. As an example, a novice user could be a researcher, who, after submitting a query through Google, has been brought to the Collection Guides Web site. This study aimed to see how easily those who had never before or had seldom used finding aids for archival research would be able to navigate the Web site, search for desired information, and submit a request for materials from a collection.

Goals for this Study

The Collection Guides Web site had been designed to present guide information by way of two separate views: a tabbed view and a standard view. The tabbed view separated categories of information about the guide into titled tabs; the user would click on the tab with a desired heading to find more specific information about this aspect of the guide. Tabs were not present in the standard view; instead, the user could navigate the guide by either clicking on the table of contents link to the desired section, or by scrolling the length of the guide. Both the tabbed view and standard view offered a search box as a means of searching for keywords throughout each guide.

The web designer had chosen the tabbed view as the default view. However, it was noted that positive feedback was given for the standard view by users familiar with archival research. The Special Collections Research Center (the stakeholders for this study) were interested in whether novice participants would prefer the tabbed view or the standard view of the collection guides. Features specific to each view were also tested and the results analyzed for their usability.

The stakeholders were also interested in seeing the participants' approach to several select features in the search results screen, such as the 'Sort by' drop-down menu, the Online Content checkbox, the 'Add to Bookbag' method of saving collections, and the left hand menu facets under 'Refine Your Search'.

Team and Partners

  • Susan Pauley, facilitator, principal investigator
  • Angie Ballard, principal investigator
  • Adam Berenbak
  • Joyce Chapman
  • Brian Dietz
  • Genya O'Gara
  • Cate Putirskis
  • Jason Ronallo
  • Linda Sellars
  • Emily Fisher Walters

Technical Details

Recordings were made in the NCSU Libraries Usability Research Lab using Morae Recorder usability testing software and analyzed using Morae Manager.

Expected Results

Several tasks produced results expected by stakeholders. Task 1 contained the most task portions that worked as predicted; tasks 1a, 1b, and 1d operated as designed, overall. For task 1a, participants did use the 'Sort by' drop-down menu to sort results. For task 1b, the majority did use the 'Date' facet under the 'Refine Your Search' menu to limit their search by date. Participants also did correctly use the 'Online Content' checkbox to limit their results to online content only under 'Refine Your Search' for task 1d.

Task 3a also worked mostly expected. Two participants first used 'Sort by' to sort the results list by creator, but then located the correct result by looking next to the Creator entry for School of Design. One participant chose 'School of Design' from the NCSU facet under 'Refine Your Search', which was an unexpected path but did lead to the correct result.

Task 5 had two portions that worked largely as expected by stakeholders and the facilitator. In task 5b, most participants used the 'Search this Collection Guide' search box to search for the keyword 'wrestling'. In task 5c, participants did click on the 'Standard View' link at the top of the screen to see the standard view of the collection guide.

Task 6a also produced expected results. Participants did use the 'Subject' facet under 'Refine Your Search' and then narrowed their searches to 'African American college students'.

Problem Areas Identified

Several major problem areas were identified as a result of conducting task 2c, task 3b/task 5d, task 5e, and task 6e.

Task 2c showed that novice users appear to have difficulty figuring out how to search the collection. Three participants used the 'Search this Collection Guide' search box, while two other participants used Cntl+F in the browser to search. However, seven participants scrolled up and down the page, scanning for the keyword 'railcars'. Surprisingly, all participants who used a targeted search method did find the correct box and folder. None of the seven participants who scrolled located this box and folder.

Task 3b and 5d were alike in that they both requested participants to search the guide more deeply for precise information. Task 3b involved searching for copyright information from the tabbed view, while task 5d involved searching from the standard view. More participants, a total of eight, were able to find this information in the tabbed view, while only three participants could do so in the standard view. This finding suggests that novice users may have greater difficulty locating precise information within the standard view.

Another problem area discovered occurred in task 5e, in which participants were asked how they would access the collection. From the standard view, participants looked for a way by which they could contact someone managing the desired materials. Some verbalized their uncertainty concerning the phrase 'Controlled Access Headings' on the page, and whether this section was somehow connected with requesting access.

A final and major problem area discovered concerned submitting requests for materials, as demonstrated in task 6e. This task also looked at whether participants would seem uncertain about or question their ability to request restricted materials. Only one participant mentioned aloud that these materials were restricted.

It was found that only five participants submitted any information through the Web form located under the 'Contact Us' tab. Information submitted was also incomplete. Only one participant had noted on her task sheet the items she wanted to request. She was able to type this information into the form. However, while she mentioned Lawrence M. Clark graduate students, she did not record any folder information, only the numbers for two of the six boxes. Of all twelve participants, none submitted all the correct information in order to request these materials.

Additional detailed participant impressions of an example finding aid, post-test survey responses, trends, and recommendations may be found in the Findings Report for this study.

Project Links

  • Script - The test script with instructions and explanations for participants.
  • Demographics Questionnaire
  • Pre-Test Questions and Answers - Questions asked about participants' impressions viewing an example finding aid, as well as complete participant responses for Question #1
  • Tasks - The list of tasks presented to participants to reference while completing tasks.
  • Post-Test Questions and Answers - Questions asked with the conclusion of tasks, as well as participant responses.
  • Thank You to Participants - Facilitator contact information and expression of thanks to participants. Distributed to participants at test session conclusions.
  • Tabbed View - Screenshot of the Collection Guides from the Tabbed View.
  • Standard View - Screenshot of the Collection Guides from the Standard View.

Reports and Presentations

Last updated: April 2010