Researching Usability

Posts Tagged ‘ISO principles

The final part of the usability inspection of selected  digital libraries examines Edinburgh University’s Aquabrowser. Hopefully at this point the patterns which emerged can now be seen. Although directly comparing each DL is not necessarily appropriate, it is possible to examine how each DL has approached similar design problems.

The full report will be presented in a slightly different format to the blogs without the separate analysis of each DL. Instead the common themes such as resource discovery, navigation, and personalisation will be discussed in relation to the DLs. Creating these five blogs has not only helped to formulate a clear structure for the full report but also formalise the findings similar to a drafting process.

Edinburgh University Aquabrowser background

The Aquabrowser technology used in this digital library was developed by the Dutch company, Medialab Solutions. Aquabrowser provides a visual search tool for digital libraries. It’s unique attribute is the ‘Discover’ system which allows users to search using a spacial 2-D animated word cloud. The technology has been adopted by many other digital libraries around the world in addition to The University of Edinburgh.


Resource discovery

Similar to WDL, Aquabrowser does not provide an advanced search form. In these cases it is because both DLs provide an alternative method of searching the site that does not rely on the search form throughout the task flow. Aquabrowser encourages users to narrow their search using tools such as the word cloud and the faceted navigation. WDL encourages users to use the world map in conjunction with the time-line.  The absence of an advanced search may have been determined by the typical user groups of each DL. Alternative search tools are often aimed at new or inexperienced users which could be the reason these have been deployed. They are also a good way for users to browse information when they do not have a set idea what they are looking for. However, if advanced or experienced users are likely to visit the DL they will expect it to have an advanced search. In order to provide accelerators for advanced users as recommended in Heuristic 7 then an advanced search should be provided.


The predominant navigation system used by Aquabrowser is their unique animated spatial 2-D word cloud in addition to more conventional faceted navigation and pagination. All of these systems were evaluated and some of the findings are below:

Animated spatial navigation

Aquabrowser uses a primary navigation system which is different from all other DLs evaluated. An animated spatial word cloud is provided to illustrate users results (see image). Associated words are offered in addition to alternative spellings and translations from other languages. A key at the bottom is supplied which explains the different colour coding. Overall this system appears intuitive to use with some minor usability findings. However, as this navigation system is unusual, user testing would provide more insight into the user’s understanding of the tool and would therefore be recommended.

Narrowing a search is possible by selecting several words in the cloud. The ‘Discovery Trail’ in blue indicates all the words selected by highlighting them to the user. However, if a user seeks to widen their search again and remove some selected words, it is not possible to do this. As a result, the user must start a new search by entering a search into the form. There does not appear to be any other method of resetting the search criteria. Consequently a user is unable to manipulate their search effectively as recommended in Heuristic 3.

Faceted navigation

Faceted navigation in the right hand column provides a wide variety of categories to narrow search results. Each category provides 5 sub sections but also provides  a ‘more’ link for the full list if there is more than 5. If a user selects links from several categories, each chosen category is removed from the navigation options. This is not immediately obvious to the user who may become confused and disorientated at the changes. In this situation the visibility of system status (Heuristic 1) could be more explicit to inform users of what’s happening. The breadcrumb navigation across the top of the search results indicates which categories have been selected. Users can narrow their search by selecting a link in the breadcrumb trail which is within a higher level within the site, but this does not provide as much flexibility as the breadcrumb trail used by WDL for example (see image).


Aquabrowser uses pagination in addition to faceted navigation in the same way as WDL and Europeana. The faceted navigation allows users to narrow the number of results, making it easier for users to move through the number of paginated results. Combining both types of navigation gives users greater control over their search as suggested in Heuristic 3. Doing so allows users to identify what they are looking for quickly and easily.

Global navigation

A traditional global navigation system is positioned across the top of the page. In Aquabrowser it is prefixed with the text ‘Go to’ which assumably is there to indicate that the links are external to the DL. However, users may scan the links quickly and not see this text (which is also less prominent than the links). Consequently users might not expect the links to lead to external sites such as the University Catalogue and Library Online. Conventionally global navigation links positioned here is used for internal navigation. As the links do not follow platform conventions (Heuristic 4) they are unlikely to meet user expectations.


Aquabrowser provides users with detailed information on the location of an item, including its shelf-mark. The shelf-mark information is linked, however when the link is selected it starts a new search. Users may expect the link to navigate to a reservation page where they can login and reserve the item prior to collection. Providing shelf-marks are important for a University library where a physical copy of the item is available. However, it is not clear to the user where a hyper-linked shelf-mark and this will lead to confusion. Re-labelling the link or adding additional wording to explain its purpose in the context of the DL would ensure that it follows real-world conventions thereby meeting users expectations as set out in Heuristic 2.

Search results

Search results are clearly presented with icons to represent the type of item and sometimes a thumbnail image of the book where appropriate. Similar to Scran, Aquabrowser provides a drop-down menu of options to sort results by author, relevance, year or title. Aquabrowser’s sort options are more visible because the user does not have to select a link to reveal the drop-down menu, unlike Scran. Providing sort options offers an additional system which gives users control over their search results as suggested in Heuristic 3 by allowing users to determine how the results are presented.

Recovery from errors when no results are returned

Help is provided to users to allow them to complete their task even when no search results are returned. Similar to Scran, suggested spelling,  advice and tips are provided. The suggested spelling is linked making it quick and easy for users to search again. However, the advice in Aquabrowser is limited and does not contain as much detailed advice or links to other resources as Scran. Doing so can help users to recover from errors quickly (Heuristic 9) and this can positively affect the overall user experience of the DL.

Social interaction, personalisation and customisation

There is currently no provision for users to interact with each other through Aquabrowser. There is also no way for users to save searches, mark or tag items of interest or create a personal folder. As Aquabrowser is often not the sole DL for an institution, it is likely that these services are provided elsewhere and are therefore not required. It would be interesting to know if users want and/or expect such services in DLs and how often they are used. This is something which will be examined in more detail when the user-centred evaluation of NeSC digital library begins later in the project.

Scran Background

Scran is a multimedia digital library providing a database of items from museums, galleries, archives and media across Scotland. It provides limited access to users for free. Those who pay a fee to subscribe can have full access to all services including the social network, Scribble.


Resource discovery

Scran provide another search feature in addition to the advanced search called ‘Fielded’ search. This appears to be similar to the advanced search but it is not made clear to the user what the purpose of this search is and how it differs from advanced search.  This does not meet Heuristic 2: Match between the system and the real world which recommends that words and phrases should be used that are familiar to the user. In addition, the links within both search forms that start a new search are not clearly identifiable. The design of the ‘Go’ link means that it does not look like a link and could be overlooked (see image). The close proximity of a ‘Clear’ link could cause users to accidentally wipe their search and force them to start again. The design of the forms are not forgiving for the user as highlighted by ISO heuristics and do not prevent potential errors from happening as recommended in Heuristic 5.Scran design


Scran also does not use faceted navigation, instead providing paginated results which users can navigate. This alternative system does not allow users to manipulate the results in the same way as other navigation systems, limiting user control (Heuristic 3). Scran does allow users to re-order results and increase the number of results presented per page. This helps to speed up time on task and therefore meets the ISO heuristics which recommend controllability and customisation.

Presentation of search results

The results are displayed in a ‘gallery’ presentation style with thumbnail images and text displayed in a grid formation. Where images are not available a replacement graphic is provided to categorise the type of information e.g. pathfinder pack. Unlike WDL and Europeana, Scran does not provide users with the option to change the view of results from gallery for list. Instead, drop-down menus are provided to allow users to alter the order of results. However, these options are hidden beneath a link titled ‘Search options’ and could therefore benefit from being more visible. Providing this service means that it meets Heuristic 3: User Control and Freedom but does not meet ISO heuristic which asks if it is clear what the user should do next.

Presentation of results in a gallery format can speed up time spent searching but DLs should be weary of a possible disadvantage to this system; the hierarchy of results is less clear, meaning that users may be unsure which results are most relevant. This can potentially conflict with Heuristic 2 (Match between the system and the real world) if the information does not appear in a logical order. Where possible users should be given to choice to display results by list if they wish.

Help and guidance

The help section within Scran can only be located via the Site map when a user is not logged in. Even for members who are logged in, the help section is difficult to find. This makes it harder for users to get help when they need it. This does not meet Heuristic 10: Help documentation which states that help information should be easy to search.  However, if a user does locate the help section, Scran provides useful video tutorials which communicate information in an engaging way and reduce the size of help documentation as also suggested by Heuristic 10.

When a search returns no results, Scran provide sufficient help for users to complete their task. Suggested spelling is provided as well as advice and tips on recovering from an error (see image). As a result, Scran provide a forgiving dialogue (ISO heuristic) which is successful in helping users diagnose and recover from errors (Heuristic 10).Scran tips

Interactive tools

Scran provide a ‘Create’ link next to each image which allows users to interact with images in a unique way. Users can manipulate the image to create a variety of media such as calendars, posters or greeting cards. A step-by-step form allows the users to step through the each option and customise their design (as recommended in Heuristic 3). Creations can then be saved to the site or downloaded in a PDF format. Providing such interactive tools helps to engage the users and allows users to customise information to suit their needs as recommended by the ISO heuristic on individualism.

Social Interaction

The online community service provided by Scran is called Scribble. It is not immediately recognisable from the title what Scribble is and there is very little information for users  to explain Scribble and how it benefits users. Limited information on labels and services means that Scribble does not meet Heuristic 4: Consistency and standards.

Access to Scribble requires a separate password to that used to log in to Scran. It currently uses the login details from ‘Stuff’ which is the service for creating calendars etc.  A link is provided to allow users to do register for Stuff if they have not already done so. However, if a user is already registered and logged into My Stuff, they are still required to log in again for Scribble. This is potentially confusing for users. By conforming to a minimalist design as recommended in Heuristic 8, a streamlined service would be created with one log in required for all Scran services.

Finally, if a user forgets their Scran ‘Stuff’ login a ‘Forgotten Password’ link is provided. However, when this is selected a message is returned that password changing has been disabled. No other information if provided to help users remember their user name and password. This makes it very difficult for those users to be able to log in to Scribble and does not meet Heuristic 9: Help users diagnose and recover from errors.

Personalisation and customisation

My Stuff is the users’ personal section of Scran and opens in a new window. This is potentially confusing for users, especially those using assisted technology such as screen readers. It can cause disorientation for those who are unaware that a new window has opened and prevent them from navigating back to the previous page. Limiting control for the users conflicts with Heuristic 3 (User control and freedom) and providing insufficient feedback to keep users informed of what is going on conflicts with Heuristic 1 (Visibility of system status).

When a user wants to save what they are working on in Scran, they must select the ‘Save’ link below the relevant item. However, when this happens a new window opens and if the user already has the Stuff section open in another window, nothing appears to happen. Users are not provided with sufficient feedback to indicate a change has happened so are less likely to know if their action was completed. Visibility of system status should be maintained to avoid confusion (Heuristic 1). It should also be made clear to users what they should do next (ISO standard: self descriptiveness).

On the whole, the  customisation tools provided by Scran are engaging and increase the usefulness of the material available.  Making some changes to improve the interface would ensure that the tools are intuitive to use and consequently encourage more members to utilise them.

Now that the report is almost finished it seemed appropriate to share some of the findings from the investigation here for feedback. It is hoped that this will help to draw conclusions from the findings and inform the subsequent usability and evaluation work on library@nesc – the digital library in development in UX2.0. Due to the size of this report, the issues highlighted here will be split into five separate blogs. The theme of each blog will be determined by each digital library (DL). The first library to be discussed is the World Digital Library.

An outline of the methodology used for this inspection can be found in a previous post. Each digital library was evaluated against a set of heuristics developed by Jakob Nielsen and Rolf Molich. In addition to these heuristics, the International Standardisation Organisation’s (ISO) set of principles from the Ergonomics of human-system interaction section (part 110) were also referenced. A useful outline of these internationally recognised principles is provided by User Focus.

World Digital Library Background

The World Digital Library (WDL) is a collaboration between the Library of Congress, UNESCO and the national libraries of 32 countries. It is offered in seven languages and provides details of rare books, maps, film clips and recordings for free. Its objective is to “promote international understanding, to expand non-English and non-western content online, as well as to contribute to research and education.”


What strikes you about WDL is the unconventional approach to search which allows users to not only search but also browse information. The home page has a world map and a time-line which are both interactive. Users can manipulate the time-line using sliders and select a thumbnail image from the map to open a carousel of images. This system is an engaging way for users to browse the library and is particularly useful to those without specific search criteria. One issue which was discovered using this system is outlined below:

  • WDLLinks within the map behave unexpectedly. If a user selects the number of items in the map they are directed to a new page of results. However, if the user selects the thumbnail image (as shown in image right), a carousel opens on top of the world map. It is not clear why links which provide the same information behave differently and there is nothing to indicate to users before selecting a link what will happen. This conflicts with Heuristic 4 (consistency and standards) within Nielsen’s set of heuristics and the ISO principles that ask ‘Does the dialogue support learning?’ and ‘Is the dialogue consistent?’

WDL have been creative in providing new ways for users to search information but what about traditional search? WDL does have a simple search form which is persistent throughout the site. Many users expect to find a search form in a digital library (or any website for that matter); therefore it is good that they have included it. However, heuristics used to evaluate WDL’s search revealed a couple of issues and are described below:

  • No advanced search. Although a basic search form is provided, a link to an advanced version is not present. The absence of an advanced search may be determined by those who use the DL. If WDL found that their core users do not use an advanced search then its presence is not important. It is plausible that users could prefer to use tools such as the map and time-line as opposed to search. However, generally an advanced search should be provided and clearly labelled to cater for all types of users. This is consistent with Nielsen’s Heuristic 7 which states that ‘Accelerators – unseen by the novice user – may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. It is also covered by an ISO standard which recommends that the dialogue is suitable for the user’s task and skill level.
  • No help provided to users when no search results are returned. A search that returns no results is commonplace in a DL which is still growing its catalogue. It is therefore important that help is given to users to allow them to successfully complete their task. Providing a list of tips that allow users to construct a more successful search is one solution. In addition, search engines which can suggests an alternative spelling of words is also very useful. Other DL’s evaluated in the report provided good examples of help when no results are returned. SCRAN for example provides links to more detailed resources on searching that provide users with some useful lessons. Heuristic 9 details this issue and requests that messages suggest a constructive solution (help users recognise, diagnose and recover from errors). The ISO standard requests that dialogues support learning and provide a forgiving dialogue reinforce this issue.

WDL 2When search results are provided, results are presented in a gallery format by default. Users have the option to switch to a more traditional list view if they wish (image above). Providing icons that are supported with text are recommended because they ensure that users understand unfamiliar graphics. This is something which WDL does successfully and fulfils Heuristic 6 which requests that options should minimise a user’s memory load and promote recognition rather than recall. It is also covered by the ISO standard which states that the dialogue should make clear what the users should do next. This system also gives users the power to customise the display of results to suit their needs and this satisfies Heuristic 3: ‘User control and freedom’.  One issue was identified regarding the display of search results which WDL should be aware of:

  • The hierarchy of search results should be clearly indicated. When a user is searching results using Gallery View, it is not clear how the results are organised. As the results are displayed horizontally as well as vertically, users may be unsure which items are most relevant. Feedback should be provided to make it clear how results are organised i.e. relevance, alphabetical, date etc. The British Library provides a good example (pictured below) of such feedback. Being aware of this issue will ensure that Heuristic 1: ‘Visibility of system status’, and ISO standard: ‘Dialogue conforms to user expectations (consistency)’ is maintained.BL

Finally many of the DL’s evaluated in the report did not adequately provide internal navigation for users between search results and item information. If a user selects an item for more detailed information there is no clear way of navigating back to the search results. As a result, users are forced to rely on the browser to navigate the site. This does not meet the guidelines of Heuristic 3: User control and freedom, which recommends that an undo and redo is provided for users. This is also the case for ISO standards which recommend controllability and self-descriptiveness.

UX2.0 is very interested in feedback on the research conducted to date. If you have a comment or question, please feel free to use the form below. The next blog will cover some of the findings from the evaluation of Europeana. bookmarks

Twitter feed