Researching Usability

Archive for the ‘Round-up’ Category

Firstly, apologies for the long gap in blog entries. My Dad passed away aged just 57 in Dec from skin cancer. Consequently I took an unscheduled break from work and with the holidays on top of that and then catching up on everything when I got back, the blog has been neglected. However, since my return I’ve manage to finalize the summative report on the heuristic evaluation which I’ve been blogging about. I’m hoping to write a separate blog with some of my conclusions as requested in the next week or so. This week’s round-up is slightly shorter than usual but normal service will resume next week!

Now that the summative report is complete, attentions have turned to the next 6 months of the project. Researching the evaluation process with regards to existing frameworks and structures is next on the agenda and will inform the evaluation of ux2 outputs. Evaluation of the technological outcomes from the project will initially be conducted next and will include persona research followed by usability and usefulness testing of the prototype.

Alan Cooper talks about qualitative research in his book ‘About Face 3: The Essentials of Interaction Design’. In the book he uses the term ‘Persona Hypothesis’ which is a good term to describe the first stage in synthesizing personas. It attempts to discuss questions such as ‘What different sorts of people might use this product?’, ‘How might their needs and behaviours vary?’ and ‘What ranges of behaviour and types of environments need to be explored?’ In the case of this project these questions relate to the type of library user who would visit library@nesc, what their needs would be and how they vary. Borrowing from existing persona research will help to generate such a hypothesis and enable recruitment for subsequent user interviews. The existing persona research being reviewed for the project is documented on the ux2 wiki.


Re-visiting the definition of a digital library

This week has been pretty busy, filled with lots of meetings and preparation for the project meeting which we are hosting on the 15th December. This week ux2 have been re-visiting the definition of the digital library and came to the (perhaps obvious) conclusion that it is something which cannot be tied down to one definitive version.  I’ve been reading ‘Evaluation of Digital Libraries: an insight into useful applications and methods’ Edited by Giannis Tsakonas and Christos Papatheodorou. The definition by Jesse H. Shera mentioned in the introduction was one which resonated with me as it seemed to touch on what we are trying to achieve in our project:

…contributing to the total communication system in society…

Though the library is an instrumentality created to maximise the utility of graphic records for the benefit of society, it achieves that goal by working with the individual and through the individual it reaches society.  (Shera, 1972:48)

Too often it feels like definitions concentrate on the technical parameters of a digital library and in differentiating it from the traditional library. This idea describes a common goal of both traditional and digital libraries; the interaction with individuals and society. Including users in the evaluation of a digital library is something which we hope to do at each stage in the project because social and individual benefits and feedback between them are important criteria to evaluate. Whatever definition used, there seem to be four critical elements which should be present in addition to the digitised format for a digital library to be correctly labelled: curation, preservation, archiving and cataloguing.

Interactive Information Retrieval (IIR)

Another term which was discussed during the meeting was Interactive Information Retrieval. It came up during the Designing User Interface tutorial which ux2 attended at ECDL09. Some of the examples discussed involved multifaceted ways of retrieving information. I started to think that there might be a better term for describing these particular interfaces because IIR can describe most forms of interaction with digital libraries from simple to complex and unique. A term was floated which might better describe IIR which uses multi-faceted/web2.0 interaction: Immersive Interactive Information Retrieval (I²R)? The dictionary defines Immersive as “pertaining to immersing or plunging into something”. I think this could describe the synchronous interaction that takes place when using web2.0 technology because the interaction is immediate and does not have to stop and start, keeping the user’s experience fluid and continuous. If there is an existing term for the type of interaction I am talking about I would be interested to find out.


For some Friday fun I thought I would share a few word clouds that I generated through the services Wordle and Tweet Cloud. I’ve known about Worlde for a while but never used it in anger. Earlier this week I heard people at the Online09 conference tweeting about the idea of using it in conjunction with a CV which seemed like a good idea. This got me thinking about it as a good way of quickly communicating information to someone to give them a snapshot of someone’s ideas and interests. I therefore decided to create one for this blog and for my delicious links to see what patterns were emerging. I’ve provided the resulting images below.

Twitter Cloud does the same kind of thing, grabbing data from all your tweets over a specified period (day/week/month/year). The clouds aren’t quite as impressive as the Wordle ones and you can’t customise the design yet but its a great idea and something which I imagine will grow in interest as people seek to analyse their tweets. As I will be marking my first anniversary using Twitter on the 9th Dec, I thought it would be appropriate to include a cloud from a year of tweets to see what it looked like. I was pleased to discover that the three most used words were: usability, thanks and blog! 🙂

Wordle blog

Wordle delicious links

Twitter Cloud: a year of tweets


You might notice that I’ve put a link to my Mendeley profile in the right-hand column of the blog recently. I finally got round to taking a closer look at it after hearing good things about it back in September. I’ve created a public folder for the ux2 project which I want to use to keep track of papers relevant to the project. For those who do not know what I’m talking about Mendeley is a network which allows researchers to create a bibliographic database (there is also a video presentation on YouTube). So far it seems to be a good way of organising papers because they are tagged and categorised for you. Recommendations aren’t available yet but I can see it being very useful and a good way to find new material.

I did run into a couple usability issues when trying to find other members and build my network. Firstly, if you want to check your email for contacts it is not possible to do so with your University email address (which is likely to be a common problem for academics). Currently you can only search Gmail, Yahoo, Hotmail, AOL and GMX. Secondly, the search system within ‘Find People’ was not as intuitive to use as I was hoping. Turns out the search form  searches your contacts by default. I did not realise this at first for a couple reasons:

  1. The search is very responsive and immediately starts searching your contacts as you type letters by highlighting where the letters appear in all of your contacts. Responsiveness is not necessarily a bad thing but I wrongly assumed that this meant it was only searching my contacts and started to think that I had to go elsewhere to search the whole system.
  2. Secondly, I did not read the message which appeared once I submitted my search and as a result, drew the wrong conclusions again (see image). I did what users often do and that is to scan the words quickly. The first part I read was ‘None of your contacts match this search term’ which only helped to reinforce my conclusion that this search form only searched my existing contacts. I therefore did not bother to continue reading the second line which said ‘Click Search to include all Mendeley users in the results’. The word ‘Search’ was linked making it a different colour and as a result it stood out from the rest of the text. Again I only read ‘Click Search’ and quickly dismissed it because I felt that I had already completed this action when I selected Search this first time. I spent quite a while looking around the rest of the site to find out how to search all members and eventually came to the conclusion that it wasn’t possible yet. To the credit of Mendeley, I tweeted my difficulties on Twitter and got a quick response. This showed how proactive they are at fixing problems. It wasn’t until I asked a colleague for help that I finally realised that the message asks users to click Search a second time to search all members. I think that it would be more intuitive for users if it did this by default. In addition it should give users the ability to choose whether they want to search all users or just their contacts, otherwise it is likely to be confusing. If something requires instructions to be used correctly it often means that it is not intuitive!

Anyway, I still believe that Mendeley has the potential to be a useful tool for researchers and I will continue to use it. Please feel free to connect with me if you also have an account.

COI Usability Toolkit

This week COI announced that they had developed a usability toolkit aimed at public sector websites (although anyone can access it). The toolkit provides good-practice guides on a variety of topics including search form design and search results design. After briefly looking at some of the guides they are effective in communicating the main points in a clear and concise manner with annotated illustrations to help time-poor users get a good understanding very quickly. There is also a section where you can test your knowledge and this helps to reinforce user’s learning. It is predominantly aimed at those with limited or no  previous knowledge of usability and avoids using any techy words or code.

To use the toolkit visit


Infomaki is an open-source, lightweight usability testing tool developed by The New York Public Library’s Digital Experience Group which I came across this week. It is based on the ideas of fivesecondtest, a tool which I have come across in the past. Based on the same premise, it asks users to answer one question. The questions can either be multiple choice or a design question asking users to state where they would click on a page to complete a specified task. You can also make comparisons between two designs and test the user’s recall of features. The beauty of the concept is that it does not require a lot of the users time. Answering one question can take only a few seconds and this has been shown to be attractive to users as the response to the survey was very high. So much so that the developers found that 90% of users wanted to answer more than one question – behaviour which is difficult to elicit through traditional market research methods.

Evidence suggests that tools like this are extremely successful in gathering a large volume of quantitative data which can help to back up one-to-one usability test data. The developers also plan to incorporate features that collect demographic data. This will add even more value to the tool as it will help in the construction of user persona’s.

Tools such as Infomaki are particularly useful to those working in digital libraries without a dedicated user research team. Open source means it is free and it can be set up by anyone interested in gathering data about their digital library.

There is more information on the software available in addition to the article on Code4Lib by Michael Lascarides which can be found here:

Usability Week, Berlin

My UX2 colleague, Boon returned for Nielsen Norman Group Usability Week in Berlin with lots of knowledge to share. One of the many things that came out of his time there was the idea of user persona creation for digital libraries. After a couple leads we were pointed to the work by Max Planck Digital Library and the persona’s they created. If anyone else knows of other work that has been conducted in persona creation for digital libraries please let us know, thanks.

Interactive Triptych Framework (ITF)

This week has been largely spent reading papers as I try to review the Interactive Triptych Framework as a possible framework for evaluating library@nesc later in the project. I still feel like I’m at the beginning of a long journey on this subject and often find that Twitter can be a massive distraction when trying to concentrate for longer than 20 minutes! That being said, I wanted to summarise the framework below from the paper which proposed it. Other papers of relevance will be documented in subsequent blogs.

Analysing and evaluating usefulness and usability in electronic information services, Tsakonas & Papatheodorou, Journal of Information Science, 2006;32

This paper presents the ITF model and suggests that interaction is effected equally by content and system characteristics. It also states that usefulness and usability are interconnected from the user’s point of view. ITF is a holistic picture of user interaction and other approaches to Electronic Information Services (EIS) evaluation. The proposed framework illustrates the interactions taking place between the three components: system, content and users. Each component interacts with each other and it is these interactions which define three evaluation approaches: usefulness, usability and performance.

An important observation of the research paper was the identification of a correlation between usefulness and usability from the users point of view. In addition to this is that fact that all attributes (usefulness, usability and performance) were equally preferred by users and considered important for their interaction. However, the research also reported that usefulness precedes usability when users are asked to state a preference suggesting that content quality is paramount for EISs.

An integrated approach which evaluates usefulness and usability is recommended in order to provide a holistic image of user interaction. If the ITF were to be adopted for the planned evaluation of library@nesc then such an approach would need to be designed. This is something which I intend to give some thought to.

Social Media event from NESTA

In addition to reading this week I also watched a streamed event on social media organised by NESTA entitled ‘Social media – a force for good?’. This well organised and interesting event invited three people with something to say on the subject to answer questions: Stephen Fry, Biz Stone (founder of Twitter) and Reid Hoffman (founder of LinkedIn). If you missed the live stream you can watch on the NESTA website. Although it was a talk on social media, Twitter inevitably dominated the questions. I’ve summarised a couple of points raised which I thought were topical and of interest:

  • Biz stated that his prediction for Twitter in years to come is that it won’t be triumph of technology, more a triumph of humanity. It seemed like there has been recent debate about social media and whether or not it is a saviour or slayer of humanity. I think scientific research will play a great role in attempting to answer this question. Twitter is relatively young and very little (if any) solid research has been done on its effects on individuals and society. There seems to be a huge amount of potential for researchers on this subject and this makes it an exciting subject.
  • Another similar question asked if things like Twitter are increasing isolation amongst people. Biz refuted this when he pointed to the many Tweetups that happen around the world. As an attendee of a weekly Twitter related meet-up- Edinburgh Coffee Morning I tend to agree. Although I am relatively new to the group it seems that many friendships have been formed through face-to-face conversation which has continued on into Twitter and more recently Facebook. The Guardian wrote this week  ‘Is local the new social now?’ and when thinking about the popularity and regularity of local Tweetups then it is fair to say that it is.
  • Finally a topic which cropped up a few times was the power of the masses on Twitter. Reid Hoffman put it as “The wisdom of the crowd or the madness of the masses.” This refers not only to the information which influences us on an individual basis but also the potential of Twitter (and all forms of mass communication) to whip people up into a frenzy with (possible) catastrophic consequences. One could argue that the masses are not on Twitter which is equally correct but I wonder how long it will be before this changes and we see more extremists using Twitter to communicate their message (if not already)? Something to be aware of at least.


I thought I would post another Dilbert strip which I saw this week. Its on the theme of cloud computing and made me laugh. Have a great weekend!

I’ve decided that since reading the excellent blog by Social Media Officer, Nicola Osborne, I have been inspired to start writing a weekly round-up of news/events/topics/discussions that have interested me. It will help me track my own research and hopefully be of interest to others.

World Usability Day (WUD): 12/11/09

Although this year’s theme was sustainability, the general purpose of WUD is to raise awareness of usability. So in that spirit I wanted to share some great usability related resources which I refer to frequently and that do a great job spreading the word:

  • UX booth: a blog with guest posts from people throughout the usability community.
  • UX blog: a blog from a usability professional, Barry Briggs who frequently posts links to other useful resources on the web.
  • IxDA: has a discussion forum on all things usability and new topics are frequently posted on Twitter to follow.
  • A List Apart: Very well known in the design community for producing great articles from some of the most prominent professionals around. Topics are not restricted to usability but also cover design, code and business.
  • UX Exchange: Relatively new to me, this is a simple UX Q&A site that aims to build a community lead, authoritative resource on the User Experience disciplines. Post a question and get an answer back fast.

I also came across a blog by Dana McKay on this year’s WUD theme which discusses the connection between usability and sustainability.

Library 2.0 Community

I joined Ning’s Library 2.0 community online yesterday and although I’ve not had much time to look through the site in detail, it does look like a promising resource for information, discussion and networking with like minded people.

BBC Digital Revolution: open source and collaborative documentary

Perhaps later than others, this week I discovered that the BBC are in the process of creating a documentary about the internet and are using a suitably interactive process to produce the 4 part series which will be aired in 2010. If you happen to be in London this weekend they have an event going on which aims to test the way the internet effects us. For more information please visit their site. It sounds really interesting and something I would love to attend if I wasn’t so far away. If like me you cant attend then there is lots of other stuff to check out on their site including film clips which you can not only watch but also download and edit. One interview which I was particularly interested in was that from Twitter creators, Biz Stone and Evan Williams. Something which they pointed out in their interview was that Twitter is essentially ‘Recipient driven communication’. This is a great term to describe Twitter and the fact that users decide what information they wish to receive and not the individual/company/corporation providing the information. This for me is one of the defining things about Twitter and the reason I think it has become so popular and consequently so powerful.

JISC mailing list: Web2

If however you still like your information pushed to you through more traditional forms such as email, a new JISC mailing list had been set up recently on the theme on Web 2 and its uses in libraries. I joined last week and have already started receiving emails with links to some useful information (one example being the Library 2.0 community above).

Twitter Re-tweet beta

This week people started tweeting about the Retweet functionality that was being trialled by Twitter and it wasn’t long before I got to test it myself. Yesterday however it was pulled by Twitter while they try to fix a bug. Anyway while it was available there seemed to be a lot of grumbling about the fact that users can’t edit their Retweets at all. This on the surface sounded crazy, the majority of people like to add their own value to Retweets and I could predict that many users would shun Twitters attempt at this service and continue using their own system through their respective client manager. However after reading an article on the subject from e-consultancy I realised what Twitter is trying to do. Quite often when users add their own comments to a tweet, character real-estate quickly becomes a problem and the users have to make an executive decision of what to delete from the original tweet. This quickly degrades tweets and consequently they become more difficult to track. Twitter could be trying to address this issue by removing this problem from tweets, leaving more room for content. However, I didn’t get the chance to try the Retweet function before it was promptly removed by Twitter so the jury is still out on this one!

Twitter Lists: filtering out all the irrelevant noise

Last week Twitter lists were the talk of the Twitterverse because they had finally been rolled out for everyone to use. At first it seemed quite novel to have a new feature but I quickly began to wonder how useful lists were in their current incarnation. Lists seem to be a way for individuals to categories groups of people they follow on Twitter, similar to groups in Tweetdeck. The group can be named anything and only be edited by the creator so if you find yourself in a list you don’t agree with there is little you can do about it, something which has been touched upon by others in the past. Additionally, if someone visits the group stream, there could be a large proportion of tweets which are unrelated to the group name. For example, although I am a usability analyst, not all my tweets are related to the subject. Thankfully I came across a blog which provides instructions on how to solve this problem by creating a hack to filter groups by a hashtag. This is a good short term solution but something I hope Twitter start to seriously think about as it could have a huge impact on the success of lists.

In plain English: Cloud Computing

Finally I wanted to share some brilliant videos which do a great job of explaining technology in laymen’s terms. They are by a company called Common Craft and their most recent edition is ‘Cloud Computing in Plain English’. They also have some other  useful videos on Twitter, Web Search Strategies and Social Media (which is a personal favourite because it uses a brilliant ice cream metaphor to explain the concept). Next week there is a free webinar from TechSoup which will be interviewing Common Craft, Mary Beth Facciolo from the Colorado State Library and Carolyn Blatchley from Cumberland County Library System in Pennsylvania to hear how they are usign these videos to support the needs of their community. If you are interested in registering from this event then sign up with TechSoup. bookmarks

Twitter feed