Researching Usability

UPA Conference 2010: Day 2

Posted on: June 3, 2010

Findings from day 2 of the UPA 2010 conference are detailed in the second part of my UPA blog.

Ethnography 101: Usability in Plein Air by Paul Bryan

Studying users in their natural environment is key to designing innovative, break-through web sites rather than incrementally improving existing designs. This session gives attendees a powerful tool for understanding their customers’ needs. Using the research process presented, attendees will plan research to support design of a mobile e-commerce application.

As our AquabrowserUX project contemplates an ethnographic study, this presentation seemed vital to better understanding the methods involved. Paul Bryan provided a very interesting insight into running such studies, explaining when to use ethnography and a typical project structure. The audience also got the chance to plan an ethnographic study using a hypothetical project.

Some basic information that I gathered from the talk is listed below:

What is ethnography?

  • It takes place in the field
  • It is observation
  • It uses interviews to clarify observations
  • It pays attention to context and artifacts
  • and it utilises a coding system for field notes to help with analysis

Some examples of ethnographic studies include:

  • 10 page diary study with 1 page completed by a participant each day
  • In home study using observation, interviews and photo montages created by participants to provide perspectives on subjects
  • Department store observation including video capture

Ethnography should be used to bring insight into a set of behaviours and answer the research question in the most economical way. It should also be used to:

  • Identify fundamental experience factors
  • Innovate the mundane
  • Operationalize key concepts
  • Discover the unspeakable, things which participants aren’t able to articulate themselves
  • Understand cultural variations

A proposed structure of an ethnographic project would be as follows:

  1. Determine research questions or focus
  2. Determine location and context
  3. Determine data capture method (this is dependent on question 1)
  4. Design data capture instruments
  5. Recruit
  6. Obtain access to the field
  7. Set up tools and materials
  8. Conduct research, including note taking
  9. Reduce data to essential values
  10. Code the data
  11. Report findings and recommendations, including a highlights video where possible
  12. Determine follow-up research

As I suspected, ethnography is not for the faint hearted (or light pocketed) because it clearly takes a lot fo time and people-power to conduct a thorough ethnographic study. It seems that as a result, only large companies (or possibly academic institutes) get the chance to do it which is a shame because it is such an informative method. For example, all video footage recorded must be examined minute by minute and transcribed. My favourite quote of the session was in response to a question over the right number of participants required for a study. Naturally this depends heavily on the nature of the project. Paul summed it up by comparing it to love: “When you know you just know”. Such a commonly asked question in user research is often a difficult one to answer exactly so I liked this honest answer.

When analysing the data collected Paul suggested a few techniques. Using the transcribed footage, go through it to develop themes (typically 5-10). In the example of a clothes shopping study this may be fit, value, appeal, style, appropriateness etc. Creating a table of quotes and mapping them to coded themes helps to validate the them. He also recommended that you focus on behaviours in ethnography, capture cases at opposite end of the user spectrum, and always look for unseen behaviours.

Designing Communities as Decision-Making Experiences by Tharon Howard and Wendy R. Howard

What can you do when designing an online community to maximize user experience? This presentation, based on two decades of managing successful online communities, will teach participants how to design sustainable online communities that attract and retain a devoted membership by providing them with “contexts for effective decision-making.”

This topic was interesting on a more personal level because it dealt with themes from my MSc dissertation on online customer communities. Tharon has recently published a book on the subject call ‘Design to Thrive’ which sounds really interesting. He and Wendy co-presented their knowledge of online communities detailing why you would create one, the difference between a community and a social network and the different types of users in a community. Their culinary acronym ‘RIBS’ (Renumeration, Influence, Belonging and Significance) provided a heuristic framework with which to follow in creating a successful community.

They pointed out that the main difference between a social network and a community is the shared purpose among members. Normally a community is developed around a theme or subject whereas social networks are created as a platform for individuals to broadcast information of interest to them and not necessarily on one topic. An online community is a useful resource when you want to build one-to-one relationships, share information quickly and easily and create a seed-bed where collective action can grow. I think this is true but that developments in social networks such as groups and categorised information means that social networking sites are beginning to provide communities within their systems.

Back to the RIBS acronym, Tharon talked about renumeration as the first heuristic for community creators. A mantra which he provided is a follows:

The most important renumeration community managers have to offer is the experience of socially constructing meaning about topics and events users wish to understand.

It is important to reward members for giving back to the community as this will reward those members, it will also ensure the continuation of the community through active participation; “It’s a two-way street“. Such rewards can include features that are ‘unlocked’ by active members and mentoring for new members (noobs). Tharon also states that influence in a community is often overlooked by managers. Members need to feel the potential for them to influence the direction of the community to continue to be an active participant. Providing exit surveys, an advisory council, a ‘report a problem’ link and rigorously enforcing published policies will help to ensure influence is incorporated into an online community.

Belonging is apparently often overlooked as well. By including shared icons, symbols or rituals  to represent a community allows members to bond through these common values and goals. Including a story of origin, an initiation ritual, levelling up ceremony, and symbols of rank all provide the sense of belonging which is important to a community member.

Significance is the building and maintenance of a community brand for those in the community. It’s a common characteristic of people to want to be part of an exclusive group. The exclusivity seems to increase the desire to join in many cases. By celebrating your community ‘celebrities’ and listing (often well-known) members in a visitor centre section of the community you can allude to its exclusivity. By making it invite only also helps to increase the significance of the community.

Touchdown! Remote Usability Testing in the Enhancement of Online Fantasy Gaming by Ania Rodriguez and Kerin Smollen

This session presents a case study on how ESPN/Disney with the assistance of Key Lime Interactive improved the user experience and increased registrations of their online fantasy football and baseball gaming through the effective use of moderated and automated remote usability studies.

This topic was the first of another series of short (this time 40 minute) presentations. As before, the time limitation often impacted on the detail within each talk. Understandably, speakers struggled to get through everything within the time allocated and either had to rush through slides or had to cut short questions at the end. Unfortunately this happened in the talk by Ania Rodriguez and Kerin Smollen. Although an interesting case of how ESPN (Smollen) have collaborated with Key Lime Interactive (Rodriguez) to conduct remote testing, it was not the type of remote testing I was hoping to learn more about. I already have some experience running an unmoderated test was more interested to hear detail on moderated remote testing. However, I was encouraged to hear that UserZoom came out favourably as the software of choice to run this remote study. I have been interested in using this software for a while and will hopefully get the chance to use it at some point in the future.

Multiple Facilitators in One Study: How to Establish Consistency by Laurie Kantner and Lori Anschuetz

In best practice for user research, a single researcher facilitates all study sessions to minimize variation. For larger studies, assigning one facilitator may miss an opportunity, such as catching select participants or delivering timely results. This presentation provides guidelines, with case study examples, for establishing consistency in multiple-facilitator studies.

Another short presentation which gave advice on how to ensure that consistency is achieved when several facilitators work on a project. It may not seem like rocket science but the best method used to capture information was a spreadsheet with various codes for observations. This document is shared and updated by each facilitator to ensure everything is accurately captured. It’s not a perfect system and often learning lessons and selecting facilitators carefully will help to reduce issues later but it seemed to work well in this case. Where most usability professionals would balk the idea of multiple facilitators which is often considered bad practice, it is too often a necessity in time constrained projects which may even be spread out around the world. Indeed Lauri and Lori suggest that multiple facilitators can bring benefits which includes more than one perspective, as the old proverb goes – ‘two heads are better than one‘!

Creating Richer Personas – Making the Mobile, International and Forward Thinking by Anthony Sampanes, Michele Snyder, Brent White and Lynn Rampoldi-Hnilo

Personas are a great way to get development teams in sync with a new space and their users. This presentation discusses solutions to extending personas to include novel types of information such as mobile behavior, cultural differences, and ways to promote forward thinking.

Examples of personas from presentation

Examples of personas from presentation

This 40 minute presentation provided lots of information that was useful to me (and the projects) and that presented new ways of working with personas. However, with additional time it would have been great to go over the data collection methods in more detail as this is something we are currently undertaking in AquabrowserUX.

Traditionally personas are limited to desktop users. However, this is changing as doing things on the move is now possible with the aid of smartphones. The presenters indicated that they found little literature on cross cultural or mobile personas which was a shock. The internationalization of business and development of smartphones is not new so I am surprised that more practitioners have not been striving to capture these elements in their own personas.

The team stated that they observed people to understand how they use smartphones to do new things, key tasks conducted, tools used, context and culture. Shadowing people over a day, surveying them and on the spot surveys, image diaries and interviews with industry people were all used to capture data. The outcome they discovered  was that mobile users are different in so many ways to other users they should therefore be considered uniquely. Consequently personas were created that focused on the mobile user, not what they did elsewhere (other personas were used for that purpose). The final personas included a variety of information and importantly, images as well.  Sections included ‘About Me’, ‘Work’, ‘Mobile Life’ and ‘My Day’.

In addition to mobile life, cultural differences were integrated throughout the persona. To incorporate a forward-thinking section called ‘Mobile Future’ researchers asked participants what they would want their phone to do in the future that it can’t currently do. This provided an opportunity for the personas to grow and not become outdated too quickly.

I hope the slides are available soon because I would love to read the personas in more detail. Outdated personas has always been a problem and was even discussed by delegates over lunch the day before. It is great to see how one organisation has tried to tackle this issue.

Usability of e-Government Web Forms from Around the World by Miriam Gerver

Government agencies worldwide are turning from paper forms to the Internet as a way for citizens to provide the government with information, a transition which has led to both successes and challenges. This presentation will highlight best practices for e-government web forms based on usability research in different countries.

Unfortunately technical issues impacted to some extend on the final presentation of the day. Fortunately the presenter had the foresight to prepare handouts of the slides which came in very handy. The bulk of the presentation was to provide insights into good and bad practices of government web forms around the globe. Some things that characterise government forms are the legal issues which require certain information to be included. For example a ‘Burden Statement’ must be provided according to US law. A Burden Statement includes information on the expected time to complete the form. Although this information is useful and should be on the page, it’s implementation in the form is not always ideal as other delegates pointed out. Position and labelling means that users may never find this information and consequently be aware that it exists.

I was impressed that some forms are designed with a background colour which matches the paper version as this helps to maintain consistency and avoid confusion. An issue raised in working with paper copies and digital forms is the potential problem of people using both simultaneously or copying work from a paper copy to the online version. By greying out irrelevant questions instead of hiding them, users can follow along with corresponding questions, avoiding potential confusion. I was also surprised to hear that some government forms allow you to submit the form with errors. If the form is important it makes sense that users are encouraged to submit it in any circumstances. However, users are also encouraged to fix errors before submitting the form where possible.

6 Responses to "UPA Conference 2010: Day 2"

[…] This post was mentioned on Twitter by User Experience 2.0, Lorraine Paterson. Lorraine Paterson said: Day 2 of my experience at #UPA2010 is now on my blog for those interested: #AquabrowserUX #inf11 #jisclms […]

[…] the original: UPA Conference 2010: Day 2 « Researching Usability var a2a_config = a2a_config || {}; a2a_config.linkname="UPA Conference 2010: Day 2 « […]

[…] więcej: UPA Conference 2010: Day 2 « Researching Usability around-the, bad-practice, balk-the-idea, idea, may-even, multiple-facilitators, projects-which, […]

[…] Einige sehr gründliche Zusammenfassungen vieler der Vorträge, die ich selbst nicht gehört habe, hat Lorraine Paterson zusammengestellt: Tag 1 Tag 2 […]

[…] this year’s UPA Conference in Munich a presentation on mobile personas highlighted the importance of making them forward thinking. The presentation discussed the dangers […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s bookmarks

Twitter feed

Error: Twitter did not respond. Please wait a few minutes and refresh this page.


%d bloggers like this: