Posts Tagged ‘theoretical models’
Information System Success Model
The next theoretical model that I look at this week is the Information System Success Model which was first developed by William H. DeLone and Ephraim R. McLean in 1992. As it says in the title, the framework was designed to create a comprehensive way of measuring the success of an information system. The premise is that “systems quality” measures technical success; “information quality” measures semantic success; and “use, user satisfaction, individual impacts” and “organisations impacts” measure effectiveness success. This was later streamlined into a revised version in 2003 which highlights the three main components of an information service: “information” (content), “system”, and “service” (see diagrams below). Each of these factors have an impact on the user; their intention to use (discussed in the TAM framework as ‘acceptance of a system’) and their actual usage of a system. These factors in turn influence user satisfaction and this provides an indication of the ultimate impact of the system on the user/group of users/organisation/industry. The net benefits can be scaled so the researcher can decide the context in which the net benefits are to be measured, keeping it useful in any situation.
There are a number of similarities between the Success Model and other models examined in this blog, including ITF and TAM. In addition to the parallels with the Success Model’s ‘Intention to use’ and ‘Use’ with TAM’s acceptance model, there are also overlaps in the Interactive Triptych Framework. Examining the system and content individually as a means of understanding the impact on the user’s behaviour (intentions and usage) is mimicked in the Usability and Usefulness of the system and content to the user in ITF. In the same vein, the usefulness and usability is also paramount when evaluating user acceptance in the Technology Acceptance Model. In this respect all three frameworks are similar. Where the Revised Success Model differs is in its application of these measurements. Where TAM evaluates if a system will be accepted by users, the Success Model can generate a list of measurable benefits which can be used to gauge the system success. This provides the opportunity to evaluate the success of a system over time, as users become more familiar with a system over time.
DeLone and McLean believe that the rapidly changing environment of information systems does not require an entirely new set of measures. They recommend that identifying success measures which already exist and have been validated through previous application can be enhanced and modified where necessary. New, untested measures should be adopted as a last resort. ITF and TAM have demonstrated similarities in their approach to rule them out as new. While TAM has received extensive testing in previous research, ITF is still relatively young. Adopting the ITF model for the UX2.0 project will hopefully further the research in this area.
Girl Geek Dinners: Edinburgh
This week I attended the 3rd Girl Geek Dinner in Edinburgh, hosted by The Informatics Forum at Edinburgh University. Girl Geeks is for women (and men!) interested in technology, creativity and computing. The speakers Emma McGrattan and Lesley Eccles provided entertaining, candid and very interesting talks on their own experiences working in technological sectors. I attended the first dinner in Edinburgh last year and noticed how successful it has become thanks to the wonderful work done by the organisers. The events attract a real mixture of professionals and students with a variety of interests. The passion in technology that everyone brings to the event always leaves me with real optimism and inspiration for the future. Long may these types of events continue.
Free Digital UX Books
For those who missed it or don’t follow me on Twitter, I came across a useful list of free user experience Ebooks compiled by Simon Whatley (link provided via @BogieZero). I personally recommend reading Search User Interfaces by Marti A. Hearst. If you know of any other free Ebooks please feel free to leave a link here or on Simon’s blog.
Power, Perception and Performance (P3)
As part of the ongoing literature review I’ve been researching some of the theoretical models created or adapted to evaluate information systems. Over the last couple of weeks I’ve been blogging about the Technology Acceptance Model (TAM) which has been used by researchers to show its effectiveness at determining user acceptance of specific systems. The paper by Andrew Dillon and Michael Morris, Power, Perception and Performance: From Usability Engineering to Technology Acceptance with the P3 Model of User Response (1999), reveals limitations of the framework in the context of a usability engineering perspective. It is not clear how well TAM predicts usage when testing prototypes as research using TAM to date has involved the testing of complete systems. If the functionality is limited or incomplete how adequately can participants rate its usefulness? Additionally, they are less likely to be able to rate the system’s ease of use if the interface features have not all been designed yet.
The data collection method was also critiqued because it relies on self-ratings from the participants. Studies have shown that user’s ratings change with repeated exposure to a system over time and that it may shift independently of the usability of the interface. This also relates to last week’s blog which suggested that what users say and what they do are not always the same. Self-ratings provide quantitive feedback from users but ideally this data should be gathered in addition to observation which is conducted at regular intervals to reflect any system self-efficacy.
The issues raised certainly do provide a strong argument against using TAM if you are a designer looking for issues to fix. TAM will tell you if a system is likely to be accepted by users but may not provide insight into why. It is more beneficial to IS professionals or managers who want to know if a system is likely to be used, for example when considering the procurement of a new IS.
The P3 model developed by Dillon and Morris uses three aspects (power, perception, and performance) to assess a user’s ability to use a system. A system’s power indicates its potential to serve the user’s tasks. Perception and performance measures the user’s behavioural reactions. Dillon and Morris believe that the P3 model predicts the capability to use a system through effectiveness and efficiency while TAM reveals the perception of the system. Consequently these different constructs make them independent entities which should not be compared: “The P3 model is an attempt at providing a unified model of use that supports both the process of design and clarifies the relationship between usability and acceptability.”
Useful program: Nutshell Mail
I was alerted to this wonderfully simple tool from Mike Coulter during his Ambition presentation, Listening Online. Trying it out is the simplest thing and takes less than a minute to set up. The website describes the program as follows:
NutshellMail takes copies of all your latest updates in your social networking and email accounts and places them in a snapshot email.
It’s a great way to manage multiple accounts and could be useful for those of you who either can’t access your social media accounts throughout the day or have so many people in your network that you find it difficult to monitor your feeds effectively. Last week I blogged about the limited usefulness of Twitter Groups because of the way they are accessed. Well I might be eating my words now because Nutshell Mail gives you the most recent results from your groups in each email along with any other accounts you choose to connect, including LinkedIn, Facebook and MySpace. You can also schedule the emails to arrive at a time that best suits you. That way its less likely to get lost amongst all the emails that wait you every morning! Although another piece of mail in your inbox might not sound like the ideal solution for some people, I’m willing to give it a try to see if it does make life a little easier.
Remote Research by Nate Bolt and Tony Tulathimutte
I was alerted to a competition this week in which UX Booth were giving away three copies of the book, Remote Research. As I’ve conducted some remote studies myself, this was a topic that interested me. I thought I would try my luck and low and behold I actually won a copy which I have already received! Books by the publisher, Rosenfeld Media are always informative – I already own Web Form Design by Luke Wroblewski and Card Sorting by Donna Spencer. Looking through the contents it looks like this book continue this trend. Most notable is the chapter entitled ‘The Challenges of Remote Testing’. The debate of remote testing versus direct testing has been ongoing for a while and looks set to continue. In this chapter some of the possible pitfalls are discussed which will hopefully help users make informed decisions on how they conduct user research and select the best tools to meet their needs. I look forward to reading this book, the simple design of Rosenfeld books makes them quick and easy to digest. Due to interest, I hope to write my own review here once I’ve finished it.
Researching Usability 