Surveying Students About Email Communications

The NC State University Libraries sends automated emails to our users when they borrow items. These emails include due date reminders, overdue notices, and billing notices. However, these emails can risk sounding impersonal, and they can induce feelings of worry or stress for users. To address this, we conducted a series of iterative user research studies over the course of updating the Libraries' email templates with a friendlier tone and relevant information. This project was the final stage. In this study, our primary goal was to evaluate the effectiveness of our newly improved emails.

In order to understand our participants’ impressions of the email templates, we developed a survey that would present them with three different types of notices: courtesy, overdue, and billing.

Here's an example of one of our email templates, which is sent to users who have just checked out a technology item on a short-term loan:

We wanted to find out how clear participants thought the emails were, how the emails made them feel, and where they found the most important information. Here is what we found:

How clear are the emails?

Participants ranked each of the email templates on a scale from 1 (very unclear) to 5 (very clear). Across all three templates, the average rating was a 4.5. Overall, the templates in this survey were highly rated for clarity. This is an indication that our updated templates are effective in communicating with our users.

How do the emails make users feel?

Participants were asked to select how they felt from this list: confident, confused, stressed, informed, respected, or other. For each of the templates, “informed” was the most frequently reported feeling with an overall average of 89.4%. This is another sign that our updates have been effective in achieving our goal of reducing user stress and confusion.

Where was the most important information found?

Participants were asked to report where they found the most important information within the emails. They were given the options of top, middle, or bottom. Overall, the majority of participants reported they found the most important information at the top of the emails. This is valuable because we want users to have quick and easy access to the most critical information.