Mon, Feb 21, 2005

Evaluating experience design

by Austin Govella

Evaluate the design of any kind of experience - online or off - using five, universal facets that describe the entirety of an experience.

Comments

One of the most limiting factors in Design isn’t the splintering of specialist groups, nor the emergence of specialised vocabularies. We lack common language for discussing Design, for communicating and evaluating the creation of experience.

We all contribute to the overall user experience, but we don’t have a clear definition for what we do: what is experience? We have fuzzy definitions. Several models are emerging (I’m working on one as well), but we still lack an objective means for evaluating experience design. We may not understand it, but we know what it looks like, what it feels like, its general shape. I think we can use several facets to evaluate the resulting user experience:

1. Personal: How well does the experience relate to the individual user? A conversation with your best friend compared to talking to the clerk at the DMV.
2. Desirable: How much do the users desire the experience? How much do they want to experience it? How much do they need it? A triple heart bypass versus having your ears pierced.
3. Enjoyable: How much do users enjoy the experience? A chore versus something you enjoy doing.
4. Accessible: How accessible is the experience for the user? How understandable, comprehendable, physically available? Climbing Mount Everest versus climbing the curb.
5. Negotiable: How able is the user to negotiate the experience to better communicate with them? How customizable?

Theoretically, you could survey a group of users to evaluate a given experience in much the same way psychological surveys are performed. A numeric value can then be given to the experience in question. Two quick examples illustrate how you can evaluate different kinds of experiences using these facets:

• Heart surgery: A very personal experience and very desirable (if you want to live), but not enjoyable.
• Solving a rubix cube: A very personal experience, and very negotiable, but not very accessible. Anyone can try, and almost everyone can manipulate the cube, but very few can solve them.

Evaluating successful experiences

A successful experience evaluates differently for different purposes. Successful sales and education require the almost perfect transmission of mental models. The better sales or education evaluate across all five facets, the more effective the sales and education will be.

Compare learning on your own to learning how to program your VCR, and you can see why so many people have learned how to do the latter. I think one can say that communicative experiences, experiences where the primary goal is to communicate, should evaluate highly on all five facets in order to be successful.

Other experiences evaluate differently for success. The success of jail time’s deterrence requires it be neither desirable, nor enjoyable. And we work at making it undesirable: restricted freedoms; small, overcrowded quarters; and a culture of violence and racism. But, in the eyes of someone trapped in a recidivist culture, jail time loses many of its deterrent features.

None of these facets are mutually exclusive. If we assign values for a given experience, there’s an interaction among these values, but no zero-sum interplay. Improvement along one facet may improve or worsen an experience’s value for another facet. And a high value in one facet might suggest high values in another, but this is not always the case.

The more enjoyable sex becomes, the more desirable the experience will be. But desirablity won’t always correlate with enjoyment. A divorce may be very desirable, but it’d be foolish to suggest it’s an enjoyable experience.

Most importantly, just as no two users ever have the same experience, these evaluations are highly subjective and can only measure the users perception of a future experience (their expectation), or it can measure their perception of the past.

Secondly, though we assign values for an experience for each of these facets, we’re not making value assessments. A successful experience does not necessarily achieve a good end. Nazi propaganda had great design, fulfilled goals, successfully transmitted mental models: it’s a stunning portfolio piece. Nazi propaganda was a successful experience even though the result was far from “good.” Tobacco advertising is another example.

Also, “accessibility” has everything and nothing to do with web accessibility. It has to do with how able I am to participate in the experience. For example, when communicating with my cat, I can enter the same room, and even pet her. She’s physically accessible, but verbal communication isn’t possible. I don’t speak Meow. Who knows what she’s saying, or what she’s thinking. We could say the same thing about my girlfriend.

Finally, these facets for experience should work independent of device or medium. Most of us work on the web, but experience happens with everything, so evaluation methods should work any where. We should use the same method to evaluate using a watch as we use for reading a novel, having sex, or ordering books from Amazon. Experience is independent of devices or objects. It happens in the head, so we need to evaluate the way a given experience interacts with what some have come to call a user’s infospace.

Other facets and models

Splitting experience into conveniently digestable bits is nothing new, and I can’t say I’ve even come close to examining every other model (check out “Forlizzi”), but of the few I’ve seen that work independent of device and medium, they inevitably conflate the mechanics of interaction with the experience that results from the interaction. I think this muddles things.

Probably the most well-known model using “facets” and “expereince” in the title is Peter Morville’s “Facets of user experience.” For the most part, the five facets I’ve mentioned here overlap with several of Peter’s seven. For example, that an experience be usable or findable, I would evaluate as accessible and negotiable. However, the center colum for Peter’s UX honeycomb attempts to evaluate the “value” of an experience. For the web and for communicating with clients, I think the honeycomb works great. For other experiences, though, I don’t think usefulness or credibility have anything to do with an experience’s value.

But my partner at Grafofini, Alex, suggests that valuability and credibility represent additional facets we should add to the list. I’m not so sure credibility can’t be reduced and evaluated using the other facets. And isn’t value an abstract conglomeration of how an experience evaluates against all five of these facets for experience?

I’m not sure I have the answer just yet, but my gut says no.

Talk About "Evaluating experience design"