Website Experience Analysis

This post explains an alternative research protocol, website experience analysis (WEA).

Website experience analysis is a research protocol (set of procedures) that can help researchers identify what specific interface elements users associate with particular interpretations.

WEA focuses on the messages that users take-away from their experience with the interface.

All interfaces try to communicate something, such as:

  • you should trust this application with your credit card data
  • you should come study for a MS degree in CGT at Purdue
  • etc.

WEA allows you to find out:

  1. whether the interface actually communicates this message – do people actually take away the message that you intended, and to what extent?
  2. what specific elements of the interface users associate with those particular messages (trust, CGT is a good program, etc.)

The WEA questionnaire is based on prominence-interpretation theory. It works with pairs of items that ask:

  1. Ratings of user perceptions (e.g. trust – on a scale of 1-10)
  2. Open-ended: what about the interface makes the user feel this way?

WEA is based on a much more complex theoretical framework of the website experience. The framework breaks the website experience down into two major dimensions: time and space. WEA then explains the phases of the experience as they unfold across time, and the elements of the website space (elements are categorized according to element functions). The theoretical framework is likely only valid for websites, because the experience with another type of interface, even though it may have the same three main temporal phases (first impression, engagement, exit) will likely differ in terms of the steps within those phases and the nature of the spatial elements and their functions.

WEA is different from a regular questionnaire because it connects perceptions with specific interface elements. Questionnaires will tell you whether the user trusts the product, but they won’t provide specific feedback as to what particular elements may account for that perception.

WEA is modular, which means that a different battery of items can be used, depending on the focus of the research. I used WEA in 2 contexts:

  1. To evaluate the experience of visiting organizational websites. Here, I used the 5 dimensions of good relationships between organizations and their publics: trust, commitment, investment, dialog, etc.
  2. To evaluate whether emergency preparedness websites persuade users to take emergency preparedness actions. Here I used a battery of items derived from a theory of fear appeals (EPPM) and assessed whether users perceived there is a threat, believe they can do something about it, believe the recommended actions would be effective, etc.

I think WEA would provide excellent feedback about how prospective students perceive the CGT department, based on their experience with the website. It would be very valuable to find out exactly what about the website makes them feel that:

  • they would benefit from a CGT MS
  • they would fit in
  • they would have a good educational experience
  • etc. – we have to determine the relevant set of items. Ideally, we would have a theory to guide item development.

WEA can be used with other research questions, such as: How do HR managers look at job candidates’ online information? (hello, Jack!)

WEA can be improved upon to better tap into emotional aspects of the user experience. It can be modified to be a more inductive approach, that elicits emotions and interpretations from users rather than asking about specific interpretations (such as trust, etc.)  – thank you, Emma, for these suggestions!

If you would like to read more about WEA, you can find the relevant citations in Google Scholar. I can provide copies of the papers if you don’t have access to them.

For CGT 512, you can consider adding a couple of WEA items to traditional usability testing in order to capture more information. Please let me know if you would like to do so and what questions you have about this.

 

Advertisements
  1. Thank you for mentioning me! I could see WEA as a good fit into UX evaluation package (which includes traditional usability testing). It focuses more on users’ subjective reactions rather than objective & quantitative assessment.

    From the inductive approaches’ perspective, usability testing can discover usability issues, while WEA might be able to reflect serious design-thinking issues based on the same set of usability issues. For example, an usability testing may uncover a problem with uploading images to a blog platform, while from users’ view, they may discern this as unprofessional and untrustworthy. But I am not sure how to guide users to express their emotions attached to certain elements without specifically probe it. May be something like: “how do you feel about this”?

    • The question is, Emma, why is it important to use an inductive approach here? Why is it needed? Or is it? Even though inductive knowledge creation is a philosophy I love, I don’t know that it is always necessary. In this case, I can think of two reasons why it may not be needed:

      1) Usually, we aim for some type of impact when designing for user experience. So, we want to evaluate if we created that particular emotion that we intended to.
      2) The spectrum of major human emotions is actually quite limited. We can capture unintended emotions by probing users to pick from a menu of major emotions – such as this one that you see on many refrigerator magnets 🙂 http://thankyourodserling.blogspot.com/2012/02/how-are-you-feeling-today.html

      • The refrigerator magnets are really funny:) But I was thinking the other day, specifically for the CGT website. In retrospective of my experiences of applying for graduate school, what impressed me of a university website was that it inspired me, and fired my dream and enthusiasm and let me feel that place was meant for me, and I belonged there. The emotions here might be inspired and enthusiastic,etc. I think different websites will have different purposes, and we can design a set of emoticons suitable for the website and also add some of the typical emotions we carry on daily such as those on the refrigerator magnets.

  1. No trackbacks yet.

Write comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: