Usability: Testing
Usability testing: "should ascertain that visitors both achieve the goal of their visit efficiently and at the same time have an enjoyable experience."
Usability must be a central concern of all website designers. In addition to dealing with specific concerns such as Web Accessibility, a website designer must consider the wide range of potential users.
Traditional Usability Testing
Usability testing is helps developers identify how people use systems and where they may encounter difficulty of use. Careful testing and revision can significantly reduce development, training and support costs. In addition, it can improve the effectiveness, efficiency, and appeal of web-based materials.
Traditional Methods Used:
- heuristic techniques such as usability lab tests, human factors experiments and prototype testing
- ethnography
- field evaluation methods
- data collection using questionnaires and interviews for task and audience analyses
Usability Testing and the User Experience
A key factor in usability testing involves accounting for the "user experience." There are many issues that must be considered in addressing the individual differences of users:
- device diversity: the web designer does not control the user's web browser, devices, operating system, connection speed, monitor size - test plans should include quality assurance across all of those listed above
- user-controlled navigation: in web design the user controls the navigation and can easily lead themselves astray
- mental models of Web usage: Nielson argues that "users don't recognize separate pages on the WWW - they see the Web as a whole and form mental models of Web usage and do not want their experience and control to differ dramatically from site to site"
- lowered switching costs: users are just a click away from something else if their experience at your site is unpleasant
- point of entry: users don't have to start at one particular page - should be able to find key information to navigate to other parts of a site or to complete key tasks
- distraction: too many pages, animations, advertisements, pop-up windows, sounds, required downloads...can lead to serious distraction and be annoying
- personalization: test to ensure how easy it is to personalize a site and how easy it is to modify personalization
- primacy of content: value of the site needs to be summarized in no more than 2 lines to keep the user's attention
- context of use: the visitor's context of use - what type of task are they trying to complete? do the content and design facilitate that task? do users find they types of interactions they expect?
- types of users to test: test the right users - do a user needs assessment prior to design and preceding the evaluation
- number of users to test: Nielson - 5 representative users will uncover the bulk of problems over the course of 3 tests
- location of test: lab or outside the lab?
- tasks to use in testing: what kind of tasks and functionality does the site support? what tasks should users be able to easily complete?
- simulating the conditions of use: give testers the ability to use the site as it's intended to be used - $$ if it is a shopping site
The Process of Web Evaluation
As you develop materials, you'll need to go through a series of evaluations.
Usabilty Evaluation Goal Setting
Identify both absolute and relative goals:
- Absolute goals are based on usability performance or satisfaction criteria that designers select for their site. Achieving the goal is judged independent of competitive sites.
- Relative goals are ones that take into account competitive sites. You compare rates of completing a task at 2 sites. For example, you might compare your website to others at the same level.
As you develop items, think about the information that will be gained. Here are two options related to "ease of use": "The web site should be easy to use" vs. "The user should perform the task without asking questions." The first option difficult to measure. The second can be easily measured.
Early Paper Testing
Before you get deep into development, consider a paper test to determine whether or not the initial designs reflect the purpose of the site. Can the user recognize the genre and purpose of the site by viewing it? Select 2-4 potential users and ask them to view a page from the site. Ask the following questions. Also look at the time it takes for them to answer the questions:
- What is the purpose of this page?
- From viewing this page, what is the purpose of the site?
- From viewing the page, can you tell me what kind of site this page belongs to?
- In viewing those two designs of the same page, which is more pleasing to you?
- Please find a given item on this page.
Storyboard Testing
When the storyboard is ready, it's time for another test. The objective is to help improve the design of the site and address the goals and functionality of the site. Mimic the site map and sketch all possible pages and connections between. You should show each page fully laid out in design details including navigation. Use 2-3 expert reviewers and potential users. Consider the following questions:
- Can you identify the purpose of this site?
- Can you name some other tasks you can perform on this site?
- Can you tell where you are in the site?
- Do you need to perform an action that is not available on the page?
- Are the textual links self-explanatory?
- Can you perform the task in three or fewer clicks starting form any page?
- Can you tell which are the on-site and which are the off-site links?
Interactive Prototype Testing
When the prototype is ready, test and determine the ease of navigation and the flow of the interface to identify problems and bottlenecks.
- Expert reviews are comprehensive (covering every relevant usability guideline) or heuristic (selecting a few high-level rules).
- Usability test are structured observations in a lab setting. The participant is given a task to perform and frustrations, problems and difficulties are noted.
Sample Usability Test
Explore the following to sample usability tests. Think about your own test situation. What would work best?
Exploratory Tasks. Open the Denver Public Library website. Examine the page and jot down your general comments about the page before starting to click. After this, do anything you would normally want to do on this page. Navigate freely around the site. Try to locate and comment on the following: Library Catalog, News, and the Teen section.
Directed Tasks. Open the Denver Public Library website. (1) You have heard that the library is available in the Spanish language. Can you find the Spanish language section? (2) You've heard that the library has some live book events. Can you find a list of the current event programs scheduled?
Read Jakob Nielson's information on Heuristic Evaluation for a great guide to his "discount usability engineering" method for quick, cheap and easy web site evaluation.
Read Jakob Nielson's Read Why You Only Test With 5 User.
Read the government report titled Research-Based Web Design & Usability Guidelines (PDF document) by Sanjay Koyanl, Robert Bailey, and Janice Nall. These current research-based guidelines address web design and usability issues. Although written with large-scale website in mind they provide excellent resources and suggestions for all web designers.
Explore the Nielson Norman Group Usability Reports. Although you can only see summaries and sample pages for free, they'll give you a good feel for the types of things professionals do during usability test!