Find Articles Usability Tests
Overview
Usability tests were conducted on three separate web site types in May-June 2005 in order to determine the best way of designing a web site to help undergraduates find articles. In Phase 1 of the Find Articles usability tests, six undergraduates were given a set of tasks related to finding articles and asked to complete the tasks using the NCSU Libraries web site before it was redesigned. In Phase 2, six undergraduates were asked to complete the same tasks using Model A, which adopted a "side-by-side" approach, and six undergraduates were asked to complete the tasks using Model B, which adopted a "flowchart" approach.
Team and Partners
Technical Details
Recordings were made in the NCSU Libraries Usability Research Lab using Morae usability testing software.
Results
Both redesigned models significantly outperformed the old web site in terms of time on task and task success, indicating that the "Find Articles" term and structure is indeed successful. In Phase 2, the "flowchart" approach of Model B was more successful than the "side-by-side" approach of Model A, because the web site itself was structured to guide undergraduates to the correct tool for their task. However, this approach ran into problems with tasks that fell outside the theoretical model of the site structure. Both Model A and Model B were constructed to differentiate between known-item and unknown-item searches, which created especial difficulties for task 4 and task 5. Citation Linker, classified as a "known-item" tool, could not be used at all for task 5, which asked students to find a known item. The testing also discovered design problems with other parts of the site, such as the E-Journal Finder search results page.
Project Links
- Tasks -- Six known- and unknown-item article-finding tasks on various topics.
- Script -- The test script with instructions and explanations for students.
- Old Site -- Screenshot of the former NCSU Libraries home page.
- Model A -- Screenshot of a Find Articles page with a "side-by-side" approach.
- Model B -- Sreenshot of a Find Articles page with a "flowchart" approach to two other pages.
- Final Design -- the Find Articles page design that was eventually implemented.
Reports and Presentations
- Spreadsheets showing that the redesigned models performed better than the old web site, and that Model B performed better than Model A.
- Problems with the old web site and suggestions for improvement.
- Usability report for Models A and B listing and discussing things users did.
- User Methods -- A table showing all the different ways users succeeded in the tasks.
- User Paths -- All user paths for all tasks on Models A and B.
Videos
Digital video recordings of the usability tests with accompanying audio commentary are also available upon request to library professionals. Contact Amanda French or any DLI staff member for a username and password to view the following videorecordings:
- HomePage.wmv (9:54, 26Mb, old site) -- The catalog search box on the old home page is too tempting.
- InfoArch.wmv (9:32, 30Mb, old site) -- Poor information architecture on the "Database Finder" and the "E-Journal Finder" pages.
- Terms.wmv (13:01, 36Mb, old site) -- Terms such as "database" and "e-journal," poor information architecture, and indistinct visual designs combine to confuse the users.
- ModelPerformance.wmv (11:16, 44Mb, Models A and B) -- A single page with too many options on it is not helpful; a site constructed with a "router" page is much more usable.
- KnownItems.wmv (9:27, 18Mb, Models A and B) -- An overly strict distinction between "known-item" and "unknown-item" searching is not a useful design principle for a library web site.
- Instructions.wmv (14:29, 48Mb, Models A and B) -- Users are task-oriented on the web and can be derailed if their path is blocked by unexpected instructions.
Note that all users shown in these recordings signed a release form that included the following language:
Recordings made during this study will be used for research and development. Therefore, I understand that my work during the test will be recorded and viewed by the staff of the NCSU Libraries. I further understand that the NCSU Libraries may wish to use segments of these recordings to illustrate presentations offered to professional audiences. I give my consent to the NCSU Libraries to use my recorded image and voice for these purposes, with the provision that my last name will not be associated with the recordings and that these recordings will not be released to any broadcast or publication media.
Last updated: November 16, 2007
|
