pr clinic

Two years ago, one of the first set of slides I posted on Slideshare was this one ~ Usability Testing: Options to Fit Any Budget. I’ve been tickled that since then it has gotten over 3300 views.

And (here’s where I brag a little) if you search Slideshare for usability testing, it is now on the first results page in position seven (yay!) a bit of a fall from its earlier spot in number one, but still ranking high enough to make me smile. I made it into a Slidecast with audio narration, and kept the script handy. So I thought it was about time I posted the pair of reference materials on my blog for your viewing pleasure.

[What follows is a script from the Slidecast on this topic found above. It has been slightly modified to remove references more appropriate to a presentation accompanied by slides]

Usability: an Essential Part of Great Web Sites

If you’ve been thinking lately that it might be time to start some usability tests on your Web site, there is a lot of things to consider. This presentation reviews three popular options that can fit any budget and gives you all the basics to get started.

Usability is an essential part of all great web sites. It is the intersection of where your content meets your technology. From a users’ perspective you are answering questions

• Are you happy or are you frustrated?
• Are you being productive or are you wasting time?

From a usability engineers’ perspective there are a variety of tools and techniques and processes to measure the answers to these questions.

User Satisfaction

Now when we talk about satisfaction and loyalty, there are really two tiers.

At the first tier, rational loyalty answers three questions.

1. Are you satisfied
2. Would you recommend it to others and
3. Would you come back?

At the higher level tier we’re talking emotional loyalty.  And here you are striking cords about confidence, integrity, pride and even passion in your product or service.

Before we go much further, we should probably acknowledge the elephant in the room.  The fact of the matter is, whether your site practices good or poor usability has an impact on visitors.

If a visitor encounters a site that is badly designed, they are going to leave. If it takes too many clicks, they are going to leave. If they are in the middle of a shopping experience or other transaction, they are going to leave.

If they don’t like what you have to offer because it is just too hard, they are going to leave. Organizations are losing because of this.

And yet organizations aren’t even adopting the simplest practices.

Why Usability Works

You are viewing this post because you know that’s a problem. What you need is the ammunition to sell usability inside your organization.

Here’s why usability works.

• It ensures that your site is going to be easy to learn
• It helps your target audience get things done
• It is going to help your users find your information, perform tasks, understand the content and accomplish their goals.  And do so without wasting time
• And it is going to bring you satisfied users who like what they see, recommend it to others and decide to return

The bottom line is you can bowl a strike when using usability.

• You get actionable results
• You don’t reinvent the wheel
• And when making changes and leveraging resources, you get to lead with facts, not opinion

When you commit to usability that means you are putting the interests of your users front and center.  Who are they? What are they like?  When do they visit your site? What do they want to achieve when they come to your site?  Fundamentally you need to understand the answers to these questions, and usability can show you how.

What’s the Right Test for Me?

There are many ways to find out which tasks your Web site visitors consider the most important and which tasks are the most popular.

You can gather research from interviews, examine your call center data, conduct surveys or field studies and even mine your search log analysis.

But you also have to make sure that you are strategic.  That means you have to assess the goals of your business and your organization.

Here you want to talk to the leadership in your organization as well as your stakeholders.  You should also look at the relevant documentation and look at your competition.

So let’s suppose I’ve sold you on the idea of doing usability testing.  You are probably wondering, what’s the right test for me?

Well you have four choices. In combination or as single shot projects they all have pluses and minuses.

1. An inspection evaluation is often called an expert review and gives you a scorecard.
2. Human performance testing has actual users clicking through your site trying to perform top tasks.
3. Software inspection is out there, we can use lift machine by usable.net to supplement our expert reviews and be another set of eyes on the data.
4. A more full throttled approach where you do card sorting and audience feedback to really get a handle on how your navigation is performing.

The Expert Review

Let’s dive into the details.  I’m going to start with expert review.

The deliverable here is a scorecard.  What i really like about the scorecard that I’ve developed is that it follows this value ladder. The idea that users start out at the bottom rung and they need to be aware before they are satisfied, they need to be satisfied before they are confident, they need to be confident before they trust and they need to be trusting before they are loyal.

Because we map back all of our indicators on the scorecard to these metrics you have a step-by-step list that prioritizes what to work on first.

Our scorecard helps you climb the ladder, so in step one it is all about awareness.  What is the page weight like? How about accessibility and marketing your site?

Next, it’s satisfaction.  Are you delivering an experience that is free from errors? If you do make a mistake, can you recover quickly?  So what is the help section like? How about those broken links?  What about search and indexes?

At a higher level you are looking at issues of confidence.

Is the site learnable? Can you find what you are looking for?  Here we look for measures of navigation, link behavior, readability and the like.

When we reach even higher on trust issues we look at privacy policy, we look at content in the about us section. We look at whether you have a review schedule for your content and are you avoiding duplication.  And do your pictures load right?  Have you optimized them for the Web?

You Get a Web Scorecard

At the end of this evaluation you get a scorecard.  Your scorecard is a roadmap to help you identify where you need to tweak, where you need to overhaul and where you are doing well. We’ve completed over 200 web site evaluations for the US House of Representatives.  Here we had a custom scorecard.

The current scorecard instrument that we’re using is a little more streamlined.  We have a total score of 100 based on 36 weighted factors and we are looking at three areas, three core processes where you need to be successful to succeed as a Web site.

• Providing valuable content
• Helping people find stuff and
• Leveraging the feedback loop

Together the 36 factors could be the basis of editorial guidelines or publishing rules.

A lot of large enterprise organizations look at a scorecard as step one in a process that’s about implementing some standards across the organization.

If you have a baseline review you get a scorecard that shows how you are performing today along with a vision of where you want to be tomorrow.  To get there you need to document the rules, so all of your content publishers know what is expected.

We have a lot of experience here and would be delighted to talk to you about helping you create publishing rules as an extra project after the scorecard is completed.

Pros and Cons

An expert review has some pros and some cons.

On the plus side, this is a method that performs really well as a change agent.  You have an opportunity to use it to pave the way to be a standards organization. Grades are a very familiar way to score things for executives, and you get a report that helps you track things over time.

On the downside it is very important that you have a uniform understanding of what the rules are, what the measured points are.  You also have to understand that this is a plus or minus score. Do you do it, or do you not do it?  There are not shades of gray in the way we do the scorecard.

Most importantly you are not involving users in this, it is an expert taking a look at your site.  And you are not really looking at task performance.

Human Performance Testing

That brings us to the next method, which is top task tune up, human performance testing.

To set the stage here, I’d like to call out Professor Barry Schwartz the author of the book Paradox of Choice, Why More is Less.   He makes the point that a lot of times people are judging their own satisfaction based on peak moments, the highs and the lows, and also the end result.

The bottom line is did they complete the task and what stood out — good and bad — along the way?

When we do a top task tune up a standard project looks something like this:

1. First you are going to be recruiting you participants, your test subjects.  No less than eight is highly recommended.
2. Then you are creating scenarios by identifying the top tasks that these folks try to do.
3. Then you conduct the test. We use the Usability Testing Environment, and we do that remotely using the go to meeting collaboration tool.
4. We are going to analyze the results for you and prepare a test report which will be very illuminating.

A quick word about our favorite tool, the applications we use to administer the test, which is called UTE.

What it does for you during a usability test is it allows you to take the test remote.  It expands the geography over which you can take the test.  It also speeds up the collection of data, and it automates the collection of data that might be in dispute.  This eliminates the guesswork of when people complete the task how many clicks it took and the like.

When you automate these sorts of things you can concentrate on analyzing and observing behavior to discover patterns that will improve your Web site.

Select the Right Tasks

Selecting the right tasks to include in your test is a critical success factor.

We recommend no more than seven. We chose the number seven because, frankly your testers are going to lose focus and get a little restless if they have to submit to a test longer than an hour long. Seven is usually the right number to assure you don’t burden your testers.

In securing your participants the thing to think about is whether they are representative of your user community.  You should know going in that there is good research out there saying you can get good results with this sort of test with as few as eight subjects.

The first time you conduct a top task tune up you will be doing it to establish a baseline of performance.  Here you are comparing the results the success ratio of your users against the objectives your organization has established about where you intends the success ratio to be

Ideally after the baseline has been set you go back and retest the same scenarios.

This way your comparison point is against the baseline. When you compare a baseline test with an iterative test you have a much more realistic grasp of the strength of the improvements made on your site based on lessons learned when doing usability testing.

Some web managers are really interested in how they stack up against the competition.  Here what you’ll plan is a top task tune up which compares how your target audience performs on your Web site versus how they perform on a competitor’s site using the same questions.

About Test Participants

Let’s switch gears a minute and talk a little bit about test participants.

You may recall that I said no less than eight participants are necessary when you do recruitment for testers.  I know of some complex site owners who decide they needed 16 participants. They did this because they had a multi-layered audience where they had a real need and desire to get into the weeds and do comparisons on how different audience members would react to the site.  Here, you would aim to split the test group between men and women, and capture info about their age and education level to match the target audience that the Web manager has in mind.

Think about what your own target audience looks like.  How would you cut the apple between gender, age and education targets?

If you do a top task tune up you’ll get a really beefy report at the end of the project which shows the success ratio of your users in performing top tasks during the test.

In addition to the success ratio we are measuring other factors such as number of clicks and time to task.

Your Test Report

In presenting the data, you might show an iterative test where you can see there was some marginal improvement between a test done in March and the test done in November, for example.  Improvements may not be as large as you would have liked, but they will be measurable results.

A test report can also be presented as a nice one-pager that you can imagine being very easy to deliver to executives. It would shows a real measurable improvement between the baseline test and a final prototype test.  I’ve seen such reports presented at a government conference, and I think its a wonderful way to showcase actionable results and show your success to your leadership.  You can include data about the success ratio, how much time it took to perform the task, the number of clicks and how they got there.

And most importantly: capture notes and observations as you listen in as the tester thinks out loud as they are trying to do the task as quickly as possible so your internal documents from the test provide as much information as possible to decision-makers.

Pros and Cons

Any usability test choice has pluses and minuses.

For the top task tune-up, on the plus side, the software is sweet. It is going to measure everything with precision.  Of course, on the downside it has to be setup correctly to collect the data.

On the plus side, the think aloud methodology is going to reveal important cues – lessons learned from the user experience.  On the downside you are missing the visual cues you’d get if you were in the room.

On the plus side, you can do this with as few as eight testers.  On the downside, you have to be really careful about selection bias when selecting those participants and not drop below eight people.

On the plus side, you are going to zero in on the top tasks the things that are really important to your users and matter most.  On the downside, you are going to miss the deeper review of content and navigation that another usability test procedure would give you.

Finally a top task tune up can be repeated over time, as we’ve seen with some of the examples I’ve shown you.

On the downside if you elect to do this only once it is going to limit the impact of choosing this method in the first place

Web Blueprint: Cardsort, Feedback and Benchmarking

The final usability test method in this presentation is what I call the Web blueprint package.

I draw that name from my Web governance model that says there are three phases of Web management- draw the blueprint, build the Web site and manage the lifecycle.    This strategic map gives you a foundation to make marketing choices to reach the right audience with the right content.

There are different times across the lifecycle of a Web site when the time may be right for doing a Web blue print. Perhaps you are at the start of a re-design and need help with navigation labels and structure. Or maybe you want to avoid an expensive redesign altogether and make targeted fixes and adjustments to poorly performing navigation. In other situations, you may need to ramp your positioning in the marketplace and get a handle on how you stack up against competitor sites from a users’ point of view.

If you go down this road, you can expect several deliverables including a card sort exercise, a facilitated feedback session and some valuable benchmarking that compares your site to your top competition.

Card-sorting 101

What is great about a card sort is you are putting your whole navigation through a rigorous exam from your users’ perspective. Does it fit the mental model of how they want the site to be organized?

You want your site navigation to be highly intuitive, simple and easy to learn, and your users can tell you how to do that.

A card sorting exercise is low cost and low tech. You are essentially putting your labels for your navigation — first tier, second tier, sometimes even third tier — all on cards in a nice big deck, and you are asking your users to sort it out and put it in a hierarchy that makes sense to them.

You are going to maximize the probability that users will find what they are looking for when you follow their mental model.

As you compare the results of different testers with the same deck of cards patterns will emerge that will improve navigation.

Remember, it is only one item per card, and you can do this as an open sort or a closed sort.  Meaning you can make some preliminary limits of the categories names you want to sort into or you can let your users decide what category names make sense to them.

Other options are to ask your participants to clarify what the words mean to them to unmask jargon.  You can also ask them to make notations about whether they understood the name and mark it on their cards.  All very revealing data.

A lot of people wonder about navigation. Should I organize my stuff by business group, by information type, by category, by subject, by process?  Let your users answer these questions by doing a card sort. It will also help you collapse parent child categories and create a simpler, cleaner and streamlined navigation. Again, intuitive to users.

Pros and Cons

When you elect to do a card sort as your usability test of choice there are pros and there are cons

On the advantage-side it is easy, it is inexpensive, it is very user-centric. And you are going to get top results to really improve the navigation structure of your site for terminology, labeling and categories.

On the downside, remember you are not doing task analysis, you are not doing best practice expert review, so you don’t get the advantages of those types of tests.

It can be a little time consuming, a little tricky to analyze and you are limited in the number of people who are your participants your testers.

But on balance card sorting can be a really good choice to get you the information you need to improve your site.

Facilitated Feedback

A facilitated feedback session is the last leg of our blueprint package.  And this is something you might elect to do yourself.  But oftentimes we notice large organizations find it very valuable to bring on help for this kind of exercise.

Let’s face it. You have enough to do managing the day to day activities of your Web site. Strategic planning is something that can be outsourced so you have an informed way to make judgments with an independent look about your organizations strategic goals in hand.

Interview key stakeholders you have identified with questions to get issues out on the table.  We like the structure of arranging a workshop or stakeholder feedback session where as a group people can get together and talk to each other about the goals, long-term objectives and what your organization wants to achieve through your Web site. What you get is a valuable, strategic report that illuminates your competitive advantage and where you stand in the marketplace.

Summary

This post has reviewed the pros and cons of a range of usability techniques.  We’ve talked about expert reviews, top task tune ups, the blueprint package, which included card sorting and facilitated feedback.

All of these can be valuable tools in your arsenal as you make the moves to make your Web site a more friendly, more satisfying and more productive for your users.

So if you are ready to move beyond exploring your options and make a commitment to usability testing, we hope you’ll give us a call. Any of the services we’ve talked about today — from expert reviews, top task tune ups to a blueprint package — can be offered as single shot projects, as iterative projects or as a combination of projects.

We think Web managers who commit to Web site usability make a smart choice. You are going to benefit the user experience, make them more satisfied, more productive and put your organization on the road to achieving its goals.