Researching Usability

Posts Tagged ‘inf11

In order to get a picture of the mobile library landscape, the mobile offering from a variety of academic institutions has been investigated. The aim of this is to understand what institutional libraries are currently doing to provide mobile services and identifying those services which The University of Edinburgh could benefit from.

North Carolina State University

NCSU have created a mobile library website which looks similar to a mobile application. The design is very similar to smart phone interfaces and this gives the website a modern look. Services which the library provide in addition to the catalogue include room booking, an instant messaging chat system with librarians and PC availability. Interestingly they also provide more unconventional services such as group finder and live webcam feeds. The group finder section helps users locate study groups in the library and the live webcams make it possible from users to monitor locations of the library remotely, including the cafe. It has been reported that the webcams generated over a third of mobile page views in the first 8 weeks that the mobile services were introduced suggesting that the webcams can be very useful to users (Meier, 2010).

Amsterdam University Library

Driek Heesakkers recently presented the case study of Amsterdam’s mobile library strategy during CILIP’s Executive briefing day, Becoming Upwardly Mobile – can libraries rise to the challenge? Amsterdam University Library‘s short-term mobile strategy resulted in a mobile web presence from a budget of €1500. This interim mobile library website provides a number of services including access to the OPAC, a simple search interface, a link to the library’s social networking page on Twitter, information on library opening hours, location, contact details and PC availability throughout the library. In addition to these basic services, the catalogue provides a simplified advanced search feature with a drop-down menu and access to the user’s library account. The interface of the search is simple with basic information in search results.

Huddersfield University and University of Bath

Huddersfield University have taken another approach to providing services on mobile devices. Using QR codes in and around their library they are able to provide links to digital information. This includes digital copies of print journals, links to mobile friendly videos and other useful digital resources. The QR codes appear at the end of book shelves, signage and on printed hand-outs. They even provide a QR code with a video on using the print credit machine (see image below). Although this technology may not be well-known among students it does have the potential to bring useful resources to students through their mobiles. As smartphones become more common place, the barrier to this technology is likely to be significantly lowered.

The University of Bath have also been using QR codes in their library. Their project explores the idea of using QR codes to help users locate items more easily by launching a map of the library on user’s mobiles. They also carried out a cross-institutional study which revealed that students are becoming more aware of QR codes and find the idea of using QR codes very appealing.

Image credit: Andrew Walsh, University of Huddersfield

Cambridge University

Cambridge University is one of the other UK university’s mentioned on M-Libraries wiki that have developed a mobile friendly site of their digital library. The mobile site focusses on allowing users to search the catalogue and access their account on the move. Notably they also highlight the benefits of logging in to the site for extra benefits including renewing items on loan, make and manage requests and build a list of items that can be emailed to any address. This would be particularly useful for research students who might otherwise write down their search results onto paper. The catalogue search is fairly simple with the addition of thumbnail images of book covers where relevant. Other useful features which are provided in the item details include a link to the Google Books preview of the full text and a map highlighting where the shelf mark is located within the library. Clearly some thought has been made to consider what information user’s need on their mobile when designing the site.

A comparitive analysis of  library mobile services in use in higher education was conducted by California Digital Library. They found that mobile library services fell into five categories 1. Library mobile website; 2. text messaging services; 3. mobile catalogue search; 4. access to resources, and 5. new tools and services. Simple information such as hours, direction, map, news and floor plans are common in many mobile library websites. Jeff Penka from OCLC supports this idea when he commented at CILIP’s Executive briefing day that “The library catalogue isn’t the only service (users) want to access via a mobile, they want to find out which computers are free, paying fines, deal with reservations, see opening hours too.”

Catalogue search is also a popular mobile service which is provided by academic digital libraries. This supports the findings from a survey conducted by Cambridge University which found that 55% of respondents were in favour of bring able to access the library catalogue from a mobile phone (Mills, 2009).

A list of resources relevant to mobile library services have been collected and compiled into a Delicious links folder. These links are open to everyone and I encourage you to add other resources I may have missed. Thanks

Last week during our regular project meetings we decided on a roadmap for the user research aspect for the last few months of the project (is it that time already?!). In the spirit of transparency I decided to publish my plan here. I’ll of course be blogging along the way, reporting findings and evaluating the work as I go. If you can point me to any existing research that I should include in my investigation, please feel free to leave a comment with a link at the end and thanks in advance!


This research will examine the current use of Web 2.0 developments in digital libraries, their use scenarios and applications. The scope of the remaining research will be centred on services which diffuse the digital library through mobile technology. The aim of this mobile research is to identify areas that may be relevant to The University of Edinburgh (UoE) and evaluate a prototype mobile library service.

Project Aim

To enhance the user experience of digital libraries through technological developments centred on usability inspection and evaluation.

Project Objectives

  1. To undertake usability inspection and contemporary UX techniques research
  2. To enhance digital libraries with state-of-the-art technologies
  3. To evaluate user experience in specific contexts involving real user communities

Mobile Research Aims:

1. Review current mobile digital library landscape, how services are diffused using mobile platforms and what UoE can learn (Obj3)

Formative evaluation informed by existing mobile digital library services and usability studies of mobile library services will be undertaken. This will help to provide a clear picture of the mobile digital library landscape which will inform the project’s own development work. Existing mobile usability research[i] will also provide insights into existing user centred design processes which can be adapted for this project.

Output: Blog series reviewing the trends in mobile digital library services, highlighting successful services and identifying what UoE can learn from other projects. In addition, a list of existing mobile digital library resources will be created as a resource for others.

2. Review good UX practices for mobile applications & websites as well as usability evaluation techniques (Obj1)

As mobile usability is a relatively new subject to the project, research will be conducted on usability practices for mobile design and development.  In addition, mobile evaluation methodologies will be identified and incorporated into the prototype evaluation (Obj2).

Output: Blog post which highlights good mobile UX resources and describes the evaluation technique which will be applied to the project.

3. Investigate what users want from a mobile library service (Obj3)

Continuing on from the Mobile Services survey conducted by Information Services (IS) in March 2010[ii], a subsequent survey will be conducted with UoE students which will focus on mobile library services. The findings will provide insight into the types of services users would find useful and this will hopefully influence the direction of development. The research will also help to support the ongoing mobile services development by IS as it will provide additional data which can be benchmarked against their previous survey.

The quantitative data gathered from the survey will be supplemented with a focus group with those likely to be using the service (end users) and those helping to provide the service (staff, developers, librarians). The findings will not only help to qualify the findings from the survey but also provide a broad perspective of how a digital library service should be shaped by including all stakeholders.

Output: Survey report detailing findings and outcomes from first focus group with stakeholders.

4. Evaluate the usability of the prototype mobile library service (Obj1)

The usability of the higher fidelity prototype will be evaluated with representative users. These one-on-one sessions will take place with a small number of users (6-12) and will be conducted using a simulator, a smart phone or the user’s own mobile handset. Qualitative date will be captured and reported. The objective of this usability study is to ensure the success of the prototype and provide a use case for digital library services at the University of Edinburgh and beyond.

Output: Summary of findings from focus group and detailed usability test report.

Workpackage 2 (WP2) within the UX2 project plan focuses on usability and UX research through a variety of methods. Part of the WP2 plan includes a usability inspection of the National e-Science Centre (NeSC) digital library. In addition to the NeSC library interface, developments within WP4 (Concentration UX Enhancement) means that it has been possible to test a prototype system alongside NeSC library and make some comparisons. The differences between each system are essentially based on the version of Blacklight they are using; NeSC is a developed system based on an older version of Blacklight while the prototype uses the latest version, although it has not undergone any additional development yet.

An opportunity arose to conduct rapid user testing on both systems by tagging onto an existing usability training course run regularly at the university. Attendees were asked to participate at the end of the course for 15 mins. This Ambush-style Guerilla testing leant itself well to the current state of the prototype; giving participants one task that fitted the current capabilities of the system meant that longer usability testing was not appropriate yet. It also meant that feedback could be received relatively quickly and fed back into the user centred design process to be tested on the next group of participants two weeks later. To learn more about Guerilla testing (also referred to as discount usability) please check out these excellent resources:
The Least You Can Do by Steve Krug (video presentation)

As stated, the Ambush-style Guerilla testing was planned to coincide with the university’s usability training course. The first available courses were organised to take place on the 15th  and 30th June. The course itself is provided for those with little/no experience or knowledge of usability testing and typically has around 20 attendees. Recruiting participants in this way made it much easier to do at short notice. However, one downside to this method is that you don’t know anything about who you are testing and time limitations make it difficult to capture any demographic or profile information. Although the prototypes are not guaranteed to be tested on representative users, this method of testing does highlight issues during the early design stages of development prior to testing with representative users. This makes it a valuable exercise even if it is not exhaustive or scientific.

This blog post intends to layout the test plan created for the first round of testing which took place on the 15th June and provide a snapshot of some initial findings. The intention is to run the Guerilla tests again during the next training course on the 30th June using an updated version of the prototype.

Task scenarios

The task scenarios were constructed based on the data set used in each system. The full test plan was piloted on a member of staff before conducting the Guerilla tests.

Task 1a:

“As part of your work you need to read a selection of current presentations on particle physics. Using the prototype, can you find a suitable presentation published in the last 2 years?”

Task 1b:

“A lecturer has asked you to find the most recent presentation on Grid computing for an event coming up. Can you find this information using the prototype?”

Tasks 1 a and b were alternated between each participant to ensure one system did not benefit from familiarisation in the previous task. However, in total 3 participants completed the tasks leaving an odd number.  This was due in part to a delay at the start of the training course which meant it finished later than expected. People are understandably less likely to remain past the time when they would normally leave work. Consequently, time restrictions made it difficult to recruit additional participants. However, the next training course is a morning session and will likely have a better chance of recruiting more participants.


All three participants preferred to use the NeSC library as opposed to the prototype.

Participants used the facet navigation much less in the prototype than NeSC library.

The facet navigation in the prototype did not behave as a participant expected.  They also did not realise what it was at first.

A participant was surprised when NeSC library placed selections within bread crumb trail in a different order to that which they had selected.

Facets in the prototype don’t look like expandable sections or links.

Facets did not always cope presenting results when facets were selected in different order e.g. participants expected to find relevant presentations within the most recent date when narrowing by year first. However, this could be symptomatic of the small data set used for testing purposes and would need to be testing on a larger data set.

Facets with a large number of categories e.g. Subject, were difficult to scan in the NeSC library. For example, keywords such as Grid Computing featured in more than one subject category making it difficult for the user to know which category to select.

The facet navigation moves from left hand side to right hand side once a search has been conducted using the NeSC library. This was unexpected and confusing to one of the participants.

Often participants were unable to find results from the most recent year (2009), because it was not in the top ten of results presented in the facet of the prototype. The absence of a ‘more’ button made it difficult to search all years or most recent.

Participants often sought and expected years to be presented in a chronological order and often requested that this be provided to make searching easier. bookmarks

Twitter feed