Researching Usability

Posts Tagged ‘students

While researching mobile library services, I’ve read a few research papers produced by other projects recently. Doing so has provided some guidance on our own research techniques and allow me to make comparisons with our own findings. One very comprehensive project in particular has proved quite informative: The California Digital Library Mobile Device User Research Project conducted a range of thorough research on the mobile strategy for their institution. Their Mobile Strategy Report includes details of surveys and interviews which they carried out earlier this year. The findings were used to detail barriers to mobile use and make recommendations when designing for mobile devices. I’ve decided to present some of their findings as I have found many similarities to our own findings. The design recommendations are also useful as they apply to all higher education institutions who may be thinking about developing mobile services. I do recommend you read the full report if you are interested in this area of research and design.

Designing Native Applications and Mobile Websites

There is a lot of debate on whether to design for native applications or provide a mobile website. As native applications are designed for a particular platform they work on a limited number of handsets. It also means that you may have to design a native applications for each platform separately which could be three or more if required. However, a mobile website is accessible to more mobile users as users are not required to download anything before use and you don’t need the approval of an app store like iTunes before it is available. California Digital Library (CDL) research indicates that a mobile website is currently the best option. Perhaps the website should be prioritized first and then development for a native application can follow. The report points to the book by Brian Fling, Mobile Device Design and Development which provides guidelines on the circumstances when a native application is the best route:

  • Charging for it
  • Creating a game
  • Using specific locations (though some devices are able to detect location through browser applications)
  • Using cameras
  • Using accelerometers (to detect motion or rotation)
  • Accessing file systems
  • Offline users

Research findings and recommendations:

Smartphone ownership: “Smartphone ownership will likely increase over the next couple of years.” – the findings from the surveys conducted at University of Edinburgh in March and November this year support this conclusion as they have shown an increase in smartphone ownership by 17% to 67%.

Activities on mobile devices with internet: “They are used less for academic purposes and are also not used often for uploading content.” – this is also supported by our own survey findings which revealed a similar story. Top activities on internet capable mobile devices included finding information such as directions or opening hours, email and text messaging, reading news articles and blogs but not academic content, social networking and video & music consumption.

Library resources for research on internet enabled mobile devices: “Most interviewees did not want to do actual academic research on their mobile devices. Despite this there was still interest in having the option to access library databases, catalogues and resources from mobile devices.” A great metaphor is referenced in the report which describes Internet consumption on mobile devices very well:

Desktop internet is like scuba diving, where the search can be “immersive” and “invites exploration and discovery.” Mobile internet use is closer to snorkeling, where”shallow dipping in and dipping out of content for quick checking of key content is desired.

(Hinman et al., 2008)

– Again this supports our findings which showed that half of respondents had not accessed any library services using their mobile devices.

CDL Recommendations:

  • Keep in mind that ownership of internet capable mobile devices may not translate to high usage of academic mobile tools.
  • Gear mobile sites and apps towards the mobile user who is looking for quick pieces of information (no complicated tasks).
  • Make library websites, databases and catalogues available on mobile devices, but do not provide all the functionality of desktop versions.
  • Provide easy methods to transfer research and resources to other devices, such as by email.

Reaching your mobile audience: Something that was not directly surveyed at UoE, California Digital Library found that 60% of students ranked fellow students as the most likely sources for finding out about new tools and services. Consequently spreading the word about new tools and services can be difficult. This is something for UX2 and UoE to be aware of when designing mobile library services.

CDL Recommendations:

  • Auto-detect mobile devices and automatically display mobile version of a site (though always provide obvious links back to the full version of the site).
  • Create mobile websites rather than apps whenever possible so that users do not  need to download software in advance of using it.
  • Advertise through a variety of channels, including campus or departmental emails, campus websites, library websites, and blogs.

Overall CDL design recommendations:

  • Set up testing practices and environments for the most heavily used device platforms (Apple iOS,  RIM Blackberry OS, and Android OS). Test with physical devices where possible rather than emulation environments.
  • Support mobile web access as opposed to building standalone mobile applications.
  • Adopt and maintain web analytical tools to accurate mobile device tracking and usage statistics for online services.
  • Continue to survey constituents and end-users annually or bi-annually to capture rapidly changing behaviours.

The recommendations provided are based on the research findings and are therefore very useful. Evidence supporting the creation of a mobile website first is something which we hope to explore in the focus group next year. As other platforms follow Apple’s example and provide native applications, it will be interesting to see how user behaviour and attitudes change. Web analytics are also important as forward planning can help to monitor the success of mobile library services and identify when further advertising is required. Again this is something that will be explored during the development and implementation of mobile library services within the UX2 project.

In November we ran a survey to all students at the university to investigate mobile demographics among students, Internet habits and attitudes towards mobile library services. The survey ran for two weeks and in that time 1,716 students responded. This was helped by an incentive prize draw of £50 Amazon vouchers – very attractive at this time of year.

In addition to demographic data and data on student’s mobile device, students were also asked about their mobile Internet habits and specifically their exposure to mobile university and library services. Finally they were asked which of the proposed mobile library services they would find most useful. The survey was loosely based on the research conducted by Information Services (IS) in March 2010 on mobile university services. Doing so allowed some comparisons to be made while also investigating mobile library services in-depth. Findings from the IS Survey can be found here:

The survey is part of the wider research within the UX2.0 project and relates to Objective 3 (deliverable 5.3) which is to evaluate the user experience in specific contexts involving real user communities. The quantitative data gathered from the survey will be supplemented by the focus group planned at the start of next year. The findings from the survey will help to shape the direction of the focus group which in turn will hopefully support the findings from the survey. Check back for a report on the focus group in the near future.

Some of the headline findings are provided below. The full report can be viewed here:

  • 66.5% of students surveyed own a smartphone. This is an increase of 17.3% from the IS survey in March 2010.
  • Apple iPhones accounted for 21.9% of smart handsets, followed closely by Nokia at 20% and Samsung at 15.3%.
  • 68% of students have pay monthly contracts.
  • 74% students have either a contract that provides unlimited access to the Internet or provides sufficient access to meet their needs.
  • Services which students access online most frequently (several times a day or daily) are websites in general, email and social networks.
  • Activities which students are least likely to carry out on their mobile handsets regularly included downloading content and uploading images to photo sharing networks.
  • The biggest frustrations students experience using the Internet on mobile handsets included slow or limited Internet connection speeds, poorly designed mobile websites or websites without a mobile compatible website and the limitations of using a small screen with small buttons.
  • The highest proportion of students surveyed stated that they had not tried to access library services using their mobile device.
  • The top 3 potential University library services which students would find most useful would be:
    • Search library catalogue
    • Reserve items on loan
    • View your library record – see your charges summary and which books are reserved, requested, booked and loan

It was interesting to find that over half of students surveyed had not tried to access library services on their mobile device. It does seem that students use their mobile for other University services such as student email, the MyEd student portal and Web CT as well as other general university information such as shuttle bus timetables. Perhaps a mobile optimised library website with useful and easy accessible information would encourage students to use library services on the move more often. That coupled with effective communication that these services exist for mobile devices.

Students do seem open to the idea of mobile library services being useful, with the most useful being access to library records, the ability to search the library catalogue and databases, reserve items on loan and locate a shelf mark. However, the usefulness is dependent on the implementation of such services as many students reported that websites not optimised for smaller screens can be very difficult to use.

Limitations of smaller screen:

“It is slower than a computer and the screen is too small for a full-sized website. The library should have a special mobile website which does not take long to load and is easy to navigate on a small screen.”

Implementation of mobile websites:

“Websites should be tailored in order to be suitable for rendering on small screens. Endless scrolling makes for bad usability.”

Percentage of students who rate each library mobile service as “Very useful” or “Generally useful”

Rank % Library Service
1. 93 View your library record – see your charges summary and which books are reserved, requested, booked and loaned
2. 92.5 Search library catalogue
3. 90 Reserve items on loan
4. 89 Search library databases
5. 87 Locate a shelf mark
6. 84.5 Check PC availability in library
7. 82 Request an item through inter-library loan
8. 71 View maps of libraries
9. 67 Receive alerts relating to library information or services etc.
10. 64 Library Maps & Locations using GPS – find your way around University libraries
11. 58 Library statistics i.e. top books in different categories and popular searches
12. 55 Friend Locator – see where friends are in the library and contact them to meet up
13. 54 Read reviews others have left on items in the library
14. 52 Share items that you’ve found and/or read in the library that you think others will find useful
15. 49 Rate and review items from the library bookmarks

Twitter feed