Usability test for WebVR Virtual Tour App and Tutorial

In January 2017 we conducted a usability test on WebVR Virtual Tour, a web-based application for viewing 360˚ photos, and a tutorial that allows people to develop this 360˚ virtual tour app themselves. Overall the app and tutorial were easy to use. We found a few opportunities for improvement, such as adding further instructions in the app.

Overview

WebVR Virtual Tour is a web-based application for viewing 360˚ photos. It was developed by Shalini Sejwani, engineering graduate student, and Mike Nutt. WebVR Virtual Tour is designed to be used with a mobile device and a Google Cardboard viewer. The app presents an environment where a “tourist” can navigate a 360˚ environment by turning her head to select and view immersive images. These images can be two-dimensional curved images, photospheres, panorama or 360˚ images. It is built using Mozilla’s A-frame WebVR Framework. A-frame VR framework can be used with a compatible browser on desktop, iOS, Android, the Oculus Rift, and the HTC Vive.

We created a tutorial to allow people to develop this 360˚ virtual tour app themselves. The step by step guide will help students become aware of how to use WebVR technology. It also provides a low-cost project that faculty can integrate into curricula about geography, architecture, nature, history, etc.

Goals of the usability test

  1. Test the app and make sure that it was intuitive and error free.
  2. Test the tutorial and see if it was easy to follow and language was clearly understandable.

What we found

  1. The interface of the app was intuitive but there were some errors that needed to be addressed.
  2. The tutorial was easy to follow, but required some minor changes.

Recommendations and Changes

Tutorial

  1. Add instructions to ‘commit’ on GitHub
  2. Tell the user when the tutorial is complete
  3. Making sure comments in html code are the same as tutorial language. Also that the line numbers in code match instructions.

App

  1. Add a pop up instruction pinning icons for home and close button in the view
  2. Calibrate the cursor selection time to be longer and update the instructions with the actual time on instructions page.
  3. Add a label to "start" image or consider a non-Hunt image to start the app

How We Did It

We had five participants, all NC State students. Three were graduate students, two were undergraduates, one female, and four male. They represented industrial engineering, computer science, aerospace engineering, industrial design, and the Fisheries, Wildlife, and Conservation Biology program. Each participant was asked about their prior experience with virtual reality.

Each user test was divided into an “app” and “tutorial” part: using our WebVR app with a Google Cardboard viewer and then using a tutorial to build the WebVR app themselves. Both parts had a set of tasks the user completed, and we asked follow up questions after each part. The tasks for the app were designed to judge the overall experience of using WebVR Virtual Tour to view 360˚ photos. The tasks for the tutorial were to designed to determine whether users were able to follow instructions to write WebVR code.

Research Questions

App Tasks

  1. Read the instructions and open the app on with a Cardboard viewer
  2. Read the instructions and open the image gallery
  3. Open a picture that has red chairs in it
  4. Navigate back to the gallery of images
  5. Navigate to the instruction page

Post-App questions

  1. Overall, how easy or hard was this app to use?
  2. Was there anything you found confusing or frustrating about this app?
  3. Were any of the instructions confusing? Follow up if yes: Any suggestions for rewording them?
  4. Can you think of anything that’s missing?
  5. Do you have any other observations you want to share with us?

Tutorial Tasks

  1. Download images from Google Drive and add them to the GitHub ‘assets’ folder.
  2. Add code to include the downloaded images in the gallery

Post-Tutorial Questions

  1. Was there anything you found confusing or frustrating about the tutorial? Follow up if yes: Any suggestions for rewording?
  2. Can you think of anything that’s missing?
  3. Overall, how easy or hard was this tutorial to complete?
  4. Do you have any final observations you want to share with us?

Team