Media Reports

Congress & Agencies

Good design

Testing & Metrics








The old computing is about what computers can do;
The new computing is about what people can do.


University of Maryland Research

Research on User Frustration (2002-2005)
See also Prof. Kent Norman's work on computer rage.

This work was supported by National Science Foundation Information Technology Research (PI-John Robinson, Sociology, IIS-0086143) Understanding the Social Impact of the Internet: A Multifaceted Multidisciplinary Approach (9/1/2000-9/1/2003). Jonathan Lazar’s participation was partially supported by Training Grant No. T42/CCT310419 from the Centers for Disease Control and Prevention/National Institute for Occupational Safety and Health.

1) Ceaparu, I., Lazar, J., Bessiere, K., Robinson, J., and Shneiderman, B., Determining causes and severity of end-user frustration, International Journal of Human-Computer Interaction 17, 3 (2004), 333-356. IJHCI Press Release

"Determining the Causes and Severity of End User Frustration"  (presentation slides available) details the 373 frustrating experiences encountered by 111 subjects during their use of personal computers for approximately 2.6 hours each. The applications in which the frustrating experiences happened most frequently were web browsing, e-mail, and word processing. The most-cited causes of the frustrating experiences were error messages, dropped network connections, long download times, and hard-to-find features. The time lost due to the frustrating experiences ranged from 30.5% of time spent on the computer to 45.9% of time spent on the computer. These disturbing results should be a basis for future study.

2) Bessiere, K., Ceaparu, I., Lazar, J., Robinson, J., and Shneiderman, B., Social and psychological influences on computer user frustration, In Bucy, E. P. and Newhagen, J. E. (Editors), Media Access: Social and Psychological Dimensions of New Technology User, Lawrence Erlbaum Associates, Mahwah, NJ (2004), 91-103.

Draft version: http://www.cs.umd.edu/local-cgi-bin/hcil/sr.pl?number=4410

This paper presents psychological and social perspectives on frustration in an attempt to clarify the relationships among variables such as personality types, cultural factors, goal attainment, workplace anger, and computer anxiety.  This literature review compares theoretical perspectives and develops a technology frustration model.

3) Bessiere, K., Newhagen, J. E., Robinson, J. P., and Shneiderman, B., A model for computer frustration: The role of instrumental and dispositional factors on incident, session, and post-session frustration and mood, Computer in Human Behavior (2004, online). doi:10.1016/j.chb.2004.03.015  


Frustration is almost universally accepted as the emotional outcome of a negative computing experience. Despite the wide use of the term, however, it has not been rigorously conceptualized as a factor in the study of the human–computer interface. This project sets out to explicate frustration as a pre-emotional state generated by the user's appraisal of the interface as an impediment to goal attainment, and looks at how user characteristics, such as self-efficacy, relate to it. This project employed episode report methodology to capture data from 144 computer users' reports of actual frustrating events as they took place. Diaries taken as users worked at their everyday tasks yield detailed data about the problems they encountered and included information about session length and an estimate of the time lost due to the experiences. Outcomes were measured as either situational or dispositional factors. Situational factors, having to do with specific events, predicted incident frustration. However, disposition variables, especially user self-efficacy, were much stronger, predicting incident and session frustration, and post-session mood. One surprising outcome was the failure of demographic variables as predictors of frustration.


4) Lazar, J., Jones, A., Bessiere, K., Ceaparu, I., Shneiderman, B. User Frustration with Technology in the Workplace. Proc. Association for Information Systems 2003 Americas Conference (2003), 2199-2202.

When poorly-designed computers frustrate users, their productivity, mood, and interactions with co-workers deteriorate. To learn more about the causes and effects of user frustration with computers in the workplace, modified time diaries were collected from 50 workplace users. This research-in-progress paper will discuss the research methodology, as well as preliminary findings.

5) Lazar J., Bessiere, K., Ceaparu, I., Robinson, J., and Shneiderman, B., Help! I’m Lost: User frustration in web navigation, IT and Society 1, 3 (March 2003), 18-26, available at http://www.stanford.edu/group/siqss/itandsociety/v01i03/v01i03a02.pdf

Computers can be valuable tools, and networked resources via the Internet can be beneficial to many different populations and communities. Unfortunately, when people are unable to reach their task goals due to frustrating experiences, this can hinder the effectiveness of technology. This research summary provides information about the user frustration research that has been performed at the University of Maryland and Towson University. Causes of user frustration are discussed in this research summary, along with the surprising finding that nearly one-third to one-half of the time spent in front of the computer is wasted due to frustrating experiences. Furthermore, when interfaces are planned to be deceptive and confusing, this can lead to increased frustration. Implications for designers and users are discussed.

6) Lazar, J., Jones, A., and Shneiderman, B., Workplace user frustration with computers: An exploratory investigation of the causes and severity, Behaviour & Information Technology 25, 3 (May-June 2006), 239-251.

Draft version: http://www.cs.umd.edu/local-cgi-bin/hcil/sr.pl?number=4662

When hard to use computers cause users to become frustrated, it can affect workplace productivity, user mood, and interactions with other co-workers. Previous research has examined the frustration that students and their families face in using computers. To learn more about the causes and measure the severity of user frustration with computers in the workplace, we collected modified time diaries from 50 workplace users, who spent an average of 5.1 hours on the computer. In this exploratory research, users reported wasting on average, 42-43% of their time on the computer due to frustrating experiences. The largest number of frustrating experiences occurred while using word processors, email, and web browsers. The causes of the frustrating experiences, the time lost due to the frustrating experiences, and the effects of the frustrating experiences on the mood of the users are discussed in this paper. Implications for designers, managers, users, information technology staff, and policymakers are discussed.

7) Lazar, J., Jones, A., Hackley, M., and Shneiderman, B., Severity and impact of computer user frustration: A comparison of student and workplace users, Interacting with Computers 18, 2 (2006), 187-207.

Draft version: http://www.cs.umd.edu/local-cgi-bin/hcil/sr.pl?number=4409

User frustration with information and computing technology is a pervasive and persistent problem. When computers crash, network congestion causes delays, and poor user interfaces trigger confusion there are dramatic consequences for individuals, organizations, and society. These frustrations, not only cause personal dissatisfaction and loss of self-efficacy, but may disrupt workplaces, slow learning, and reduce participation in local and national communities. Our exploratory study of 107 student computer users and 50 workplace computer users shows high levels of frustration and loss of 1/3–1/2 of time spent. This paper reports on the incident and individual factors that cause of frustration, and how they raise frustration severity. It examines the frustration impacts on the daily interactions of the users. The time lost and time to fix problem, and importance of task, strongly correlate with frustration levels for both student and workplace users. Differences between students and workplace users are discussed in the paper, as are implications for researchers.


Web Resources

These are web pages that contain online resources for students, researchers and the general public in the domain of human computer interaction.


CHARM-Choosing HCI Appropriate Research Methods

Human-Computer Interaction Resource Network

HCI Bibliography

Online Guide to Usability Resources

Usability Methods Toolbox

Usable Web

User Interface Design and Usability


The Software Usability Research Laboratory (SURL)


            Papers - Frustration

Adams, W., Brown, J., Rapeepun, D., Williams, W. (2001). The Effectiveness of Online Help Systems. Downloaded on April 9, 2002. Working paper available at: http://www.otal.umd.edu/SHORE2001/help/index.html.

Baecker, R., Booth, K., Jovicic, S., McGrenere, J. and Moore, G. (2000). Reducing the Gap Between What Users Know and What They Need to Know. Proceedings of the ACM 2000 International Conference on Intelligent User Interfaces, 17-23.

Berkowitz, L. (1978). Whatever Happened to the Frustration-Aggression Hypothesis? American Behavioral Scientist, 21(5), 691-708.

Bias, R., and Mayhew, D. (1994). (eds.) Cost-Justifying Usability. San Francisco: Academic Press.

Brosnan, M. (1998). The Impact of Computer Anxiety and Self-Efficacy Upon Performance. Journal of Computer Assisted Learning, 3(14), 223-234.

Carroll, J.,and Carrithers, C. (1984). Training Wheels in a User Interface. Communications of the ACM, 27(8), 800-806.

Collins, C., Caputi, P., Rawstorne, P., & Jayasuriya, R. (1999). Correlates of End-User Performance and Satisfaction with the Implementation of a Statistical Software Package. Proceedings of the 10 th Australasian Conference on Information Systems.

Compaq, Inc. (2001). Rage Against the Machine – a Compaq survey. Downloaded on: April 9, 2002. Available at: http://www.compaq.presscentre.co.uk/corp/Releases/release.asp?ReleaseID=485&NID=Research

Davis, F. (1993). User acceptance of information technology: system characteristics, user perceptions, and behavioral impacts. International Journal of Man-Machine Studies, 38, 475-487.

Dollard, J., Doob, L., Miller, N., Mowrer, O., & Sears, R. (1939). Frustration and Aggression. New Haven: Yale University Press. 

Freud, S. Types of Onset of Neurosis, in James Strachey, ed. (1958). The Standard Edition of the Complete Psychological Works of Sigmund Freud vol. 12. London: Hogarth Press.

Jacko, J., Sears, A., & Borella, M. (2000). The effect of network delay and media on user perceptions of web resources. Behaviour and Information Technology, 19(6), 427-439. 

Johnson, C. (1995). Time and the Web: Representing and Reasoning about Temporal Properties of Interaction with Distributed Systems Time and Space. Proceedings of the HCI'95 Conference on People and Computers, 39-50.

Johnson, C. (1998). Electronic Gridlock, information saturation, and the unpredictability of information retrieval over the World Wide Web. In P. Palanque and F. Paterno (eds.) Formal Methods in Human-Computer Interaction (pp. 261-282) London: Springer.

Lazar, J. (2001). User-Centered Web Development. Sudbury, MA: Jones and Bartlett Publishers.

Lazar, J. and Huang, Y. (2003,in press). Improved Error Message Design in Web Browsers. In J. Ratner (ed.). Human Factors and Web Development (2 nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.

Lazar J., and Norcio, A. (2002). Novice User Perception of Error on the Web: Experimental Findings. Paper under review.

Lazar, J., & Norcio, A. (2000). System and Training Design for End-User Error. In S. Clarke & B. Lehaney (Eds.), Human-Centered Methods in Information Systems: Current Research and Practice (pp. 76-90). Hershey, PA: Idea Group Publishing.

Lazar, J., & Preece, J. (2001). Using Electronic Surveys to Evaluate Networked Resources: From Idea to Implementation. In C. McClure & J. Bertot (Eds.), Evaluating Networked Information Services: Techniques, Policy, and Issues . Medford, NJ: Information Today, 137-154.

Lloyd, B.H. & Gressard, C. (1984). Reliability and Factorial Validity of Computer Attitude Scales. Educational and Psychological Measurement, 44, 501-505.

Murphy, C., Coover, D., & Owen, S. (1989). Development and Validation of the Computer Self-efficacy Scale. Educational and Psychological Measurement, 49, 893-899.

Nash, J.B. & Moroz, P.A. (1997). An examination of the factor structures of the Computer Attitude Scale. Journal of Educational Computing Research, 17(4), 341-356.

Norman, D. (1983). Design rules based on analyses of human error. Communications of the ACM, 26(4), 254-258.

Olaniran, B. (1996). A Model of Group Satisfaction in Computer-Mediated-Communication and Face-to-Face Meetings. Behavior and Technology, 15(1), 24-36.

Ramsay, J., Barbesi, A., & Preece, J. (1998). A psychological investigation of long retrieval times on the World Wide Web. Interacting with Computers, 10, 77-86.

Riseberg, J., Klein, J., Fernandez, R., and Picard, R. (1998) Frustrating the User On Purpose: Using Biosignals in a Pilot Study to Detect the User's Emotional State. Proceedings of ACM 1998 CHI: Conference on Human Factors in Computing Systems, 227-228.

Schleifer, L. and Amick, B. (1989). System Response Time and Method of Pay: Stress Effects in Computer-Based Tasks Articles. International Journal of Human-Computer Interaction 1(1), 23-39.

Sears, A., & Jacko, J. (2000). Understanding the relation between network quality of service and the usability of distributed multimedia documents. Human-Computer Interaction, 15(1), 43-68.

Shubin, H., and Meehan, M. (1997). Navigation in web applications. Interactions, 4(6), 13-17.

Shneiderman, B. (2000). Universal Usability: Pushing Human-Computer Interaction Research to Empower Every Citizen. Communications of the ACM, 43(5), 84-91.

Shneiderman, B. (1998). Designing the User Interface: Strategies for Effective Human-Computer Interaction. (3rd ed.). Reading, MA: Addison-Wesley.

Shneiderman, B. (1982). System message design: Guidelines and experimental results. In A. Badre and B. Shneiderman (eds). Directions in Human/Computer Interaction. Norwood, NJ: Ablex Publishing, 55-78.

Zviran, M. (1992). Evaluating User Satisfaction in a Hospital Environment: An Exploratory Study. Health Care Management Review, 17(3), 51-62.












Web Accessibility