Researching Usability

Posts Tagged ‘SWOT

Iterative evaluation process

Last month saw the project plan and conduct a contextual enquiry as part of the data gathering process for persona development. This field study involved gathering data from Edinburgh University library users in situ. The aims of the exercise was to understand the background of visitors, their exposure to Aquabrowser and their information seeking behaviour.

Two site visits were conducted on the 11th and 12th May in the main library at George Square. In addition to this longer (45 minute) user interviews are scheduled to take place starting in the week beginning 14th June (next week).

Below is a brief evaluation of this research phase. Detailed results will be published in a subsequent blog post.

SWOT analysis:


Recruitment of participants for longer interviews is going well. At last count we had 74 respondents from a selection of backgrounds including undergraduates, post graduates, PhD students, staff and librarians. Participant availability is also spread over June, August and September meaning that recruitment looks achievable.

We have reviewed the timing of the usability testing to take into account the persona work. Testing will now take place in two stages: 1. Aug and 2. Sept to capture data from 1. staff (when university is quiet) and 2. students (to increase availability during freshers week). This strategy will ensure that the persona research can be used effectively to recruit representative users.

By using a variety of resources, a master interview script has been created that will then be altered to suit different groups: students, staff, librarians. The interview will be piloted before the first interview takes place allowing any final changes to be made beforehand. The interviews themselves will be conducted in pairs to begin with, allowing the interviewer to concentrate on their questions while someone else takes rigorous notes. Doing so will also ensure no information is missed.


The contextual enquiry approach in the library has been limited by a number of factors. Timing of the exercise meant that meeting a range of library users was difficult. Exams were happening during this time meaning that the main library users were students studying. Consequently these students were very busy, stressed and engrossed in work for a large part of their time in the library. Trying to approach students to interview was therefore limited to those who were wandering around the main foyer. In addition, one of the biggest barriers to observing user behaviour of Aquabrowser was the limited awareness of it among students. Users currently have the choice of two catalogues, Voyager  and Aquabrowser. As a result, very few observations of students using Aquabrowser naturally were made.

A diary study has been proposed to complement the library observations and user interviews. However, the limited timescale and budget of the project will make it more difficult to recruit a willing participant. In addition, the level of resources required to run and manage such a study could be difficult  as it would be required to run alongside other ongoing work within the project and for the UX2.0 project.


After meeting with engineer, Meindert from Aquabrowser, several opportunities have presented themselves. There is a real possibility of accessing services not currently implemented by Edinburgh University though a demo site and other University libraries. This will allow us to better understand ‘My Discoveries’ and observe how social services including user-generated ratings and reviews are used. It will be possible to demonstrate these services to University users in order to gauge acceptance of such technology and perhaps create a case for its implementation.


Participant cancellations and no shows during interviews are always a threat in user research but with an extensive list of willing and pre-screened participants, finding replacements should not be a problem.

The scope of the project was narrowed after realising that was too wide to be evaluated in full. Narrowing the scope from ‘library in general’ to ‘digital library catalogues’ means the evaluation is more achievable within the timescale. bookmarks

Twitter feed