User Frustration with
Technology in the Workplace
Jonathan
Lazar1, Adam Jones1, Katie Bessiere2, Irina
Ceaparu3, and Ben Shneiderman3
1Department of Computer and
Information Sciences,
&
Center for Applied
Information Technology,
& Universal Usability Laboratory
Towson University, Towson,
Maryland, 21252
2Human-Computer Interaction
Institute
Carnegie-Mellon University
Pittsburgh, PennsylvaniaA, ??????15213
3Department of Computer Science, Human-Computer Interaction
Laboratory,
Institute for Advanced
Computer Studies & Institute for Systems Research
University of Maryland,
College Park, Maryland 20742
E-mail: jlazar@towson.edu; ajones5@towson.edu; katieb@cmu.edu; irina@cs.umd.edu; ben@cs.umd.edu
Keywords
Frustration, Usability, Technology
Acceptance, Cost justification, Interface design, User satisfaction,
Human-Computer Interaction
Abstract
When
hard to use computers that are hard to
use cause users to become frustrated, itthis can affect workplace
productivity, user mood, and interactions with other co-workers. Previous
research has examined the frustration that graduate students and their families face in using computers.
To learn more about the causes and effects of user frustration with computers
in the workplace, we collected
modified time diaries were collected from 50 workplace users, who spent an average of 5.1
hours on the computer. In this experiment, users reported wasting on average, 42-4328-35% of their
time on the computer due to frustrating experiences. The causes of the
frustrating experiences, the time lost due to the frustrating experiences, and
the effects of the frustrating experiences on the mood of the users are
discussed in this paper. Implications for designers, managers, usersinterface designers, and employees a, information technology staff,
and policymakers are discussed.
Introduction
A September 2001 recent report released byof the National
Telecommunications and Information Administration reports that as of September
2001, 56.7% adults (employed and over age 25) in the United States use a
computer in the workplace (NTIA,
2001). In addition, 81% of those employed in
managerial and professional jobs, and 71% of those in technical, sales, and
administrative support jobs utilized computers as part of their work
environment. This indicates that in
‘white-collar’ jobs, computer use is becoming very prevalent.
With the rising ubiquity of computer usage in
American society in the home, school, and workplace, research has begun to
focus on the possible consequences of such use. Research on computer anxiety, attitudes, and frustration has
shown that a disturbing portion of computer users suffer from negative affective
reactions towards the computer, which can subsequently affect whether or not
they use the computer, and whether or not they use the computer
effectively. Research on frustration,
both in individuals and organizations, has shown that frustration can lead to
maladaptive behaviors that can subsequently lower effective goal-oriented
behavior. In addition, research has
shown that between one third and one half of the time spent in front of the
computer was lost due to frustrating experiences -- when considering both the
time it took to fix the problem and any additional time that was lost due to
the problem (Ceaparu,
Lazar, Bessiere, Robinson, & Shneiderman, 20032).
Because computers are so prevalent in
organizations, it is important to examine the role of computers in the
organization and the possible consequences arising from their use. In this experiment, 50 workplace users
recorded their frustrations with computers through the use of modified time
diaries. There are solutions to the causes of user frustration.—these are not impossible to solve.
However, the first step is to understanding
the causes themselvesof user
frustration, which can lead to experimental testing of improved
interfaces to address these frustrations, and then implementation of these
solutions in industry. Computers play an important role in affecting the
performance of individuals within organizations, therefore, this research
should be of great interest to businesses and other organizations, because
improved interfaces can improve the bottom-line and corporate profit.
Background Research
Frustration
Frustration is often
defined in different ways,
making the subject itself somewhat ambiguous creating an ambiguity that surrounds the term. Frustration was first introduced by Sigmund
Freud as a concept both external and internal in nature and related to the
concept of goal attainment. Frustration
occurs when there is an inhibiting condition which interferes with or stops the
realization of a goal. All action has a
purpose or goal whether explicit or implicit, and any interruption to the
completion of an action or task can cause frustration. For Freud, frustration included both
external barriers to goal attainment and internal obstacles blocking
satisfaction (Freud, 1921).
This concept of frustration as a duality is continued in the analysis of frustration as both cause and effect (Britt & Janus, 1940). As a cause, frustration is an external event, acting as a stimulus to an individual and eliciting an emotional reaction. In this case, the emotional response is the effect, and the individual is aroused by this external cause and a response is often directed towards the environment.
Dollard et al. (1939) define
frustration as “an interference with the occurrence of an instigated
goal-response at its proper time in the behavior sequence” (Dollard, Doob, Miller,
Mowrer, & Sears, 1939). Because an instigated goal response entails only
that the goal be anticipated, frustration is due to the expectation and
anticipation of a goal, not the actual attainment of the goal (Berkowitz, 1978). If the goal is unfulfilled, frustration is experienced because
satisfaction was not achieved and hopes were suddenly thwarted. The thwarting or hindrance -- terms often
used synonymously with frustration -- is not limited to the actual activity in
progress, but relates to what the individual is expecting (Mowrer, 1938a).
Frustrations, in all cases, are aversive events (Ferster, 1957) having as their main defining feature the element of a barrier or obstruction. This barrier can take the form of an actual barrier, or an imaginary one such as the response to anticipated punishment or injury (Mowrer, 1938b). A frustrating situation, then, is defined as any “in which an obstacle – physical, social, conceptual or environmental – prevents the satisfaction of a desire” (Barker, 1938). These blocks to goal attainment may be both internal and external (Shorkey & Crocker, 1981), similar to the duality proposed by Freud. Internal blocks consist of deficiencies within the individual such as a lack of knowledge, skill, or physical ability. External blocks could include the physical environment, social or legal barriers such as laws or mores, or the behavior of other people.
Factors Affecting Level of Frustration
The level of frustration
experienced by an individual clearly can differ depending on the circumstances
surrounding the frustrating experience and on the individuals themselves. One major factor in goal formation and
achievement is goal commitment, which refers to the determination to try for
and persist in the achievement of a goal (Campion & Lord,
1982). Research on goal theory indicates that goal
commitment has a strong relationship to performance and is related to both two factors: 1) the importance of the task
or outcome and 2) the belief that
the goal can be accomplished (Locke & Latham,
2002).
Individuals will have a high commitment to a goal when the goal is important to them and they believe that the goal can be attained (Locke, 1996). The importance of the goal, in addition to the strength of the desire to obtain the goal (Dollard et al., 1939), will affect the level of goal-commitment as well as the strength of the subsequent reaction to the interruption. Self-efficacy, the belief in one’s personal capabilities, can also affect goal commitment (Locke & Latham, 1990) in that the belief about how well a task can be performed when it involves setbacks, obstacles, or failures may affect how committed individuals are to that goal (Bandura, 1986).
Judgments of efficacy are related
to the amount of effort expended, how long they persist at the task, and
resiliency in the case of failure or setback (Bandura, 1986, 1997b). Self-efficacy influencesaffects emotional
states as well; how much stress or depression people experience in difficult
situations is dependent on how well they think they can cope with the situation
(Bandura, 1997a). The level of frustration that people experience, therefore, would
be affected
influenced by
how important the goal was to them, as well as how confident they are in their
abilities. “Because goal-directed
behavior involves valued, purposeful action, failure to attain goals may
therefore result in highly charged emotional outcomes,” (Lincecum, 2000) including frustration.
Cultural factors can also play a role in the level of frustration experienced by individuals when coming across obstacles to their path of action. Social Learning Theory (Bandura, 1973) states that “rather than frustration generating an aggressive drive, aversive treatment produces a general state of emotional arousal that can facilitate a variety of behaviors, depending on the types of responses the person has learned for coping with stress and their relative effectiveness” (p. 53). Ways of coping with frustration are therefore learned from the society and are governed and constrained by the laws of a society. This can contribute to the level of frustration tolerance that individuals have, which is also affected by their prior experience and task specific self-efficacy.
According to Freud, it is not simply the nature of the frustrating incident that determines how people will react to it. Rather, there is an interplay between the situation and the psychological characteristics of individuals. The level of maturity of the individual also plays a part in the reactions to frustration (Barker, Dembo, & Lewin, 1965). With maturity, there is an increase in the variety of responses to a situation employed by individuals, in the control of the environment, and in their ability to employ problem-solving behavior and plan steps to obtain the goal. It would appear that learning, which is culturally determined, is a major factor in developing socially acceptable responses to frustration.
Two additionalfinal
factors that may influenceaffect
the force of the frustration are the severity of the interruption and the
degree of interference with the goal attainment (Dollard et. al. 1939). All obstructions are not equally frustrating,
and the severity and unexpectedness of the block will also factor into the
strength of the response. In addition,
if individuals perceive that the thwarting was justified by socially acceptable
rules, as opposed to being arbitrary, the frustration response may be minimized
(Baron, 1977). This may be due to the lowering of expectations because of extra
information available to the individual. As stated above, it is the anticipation
of success that affects frustration, and not the actual achievement of the
goal. Therefore, if individuals expect
to be thwarted or have a low expectation of success, frustration may be
minimized.
Responses to Frustration
The responses to
frustration by individuals can be either adaptive or maladaptive (Shorkey & Crocker,
1981). Adaptive responses are constructive and are
implemented to solve the problem that is blocking goal attainment. They may include preemptive efforts to avoid
the problemblock,
or once the block problemi is encountered, problem solving
strategies to overcome or circumvent the problem. Freud lists two types of adaptive responses: 1) transforming stress into active energy
and reapplying this energy towards the original goal, and 2) identifying and
pursuing alternative goals. Maladaptive
responses, on the other hand, are characterized by a lack of constructive
problem solving and often make the frustrating experience worse by creating
additional problems. These maladaptive
responses may be further categorized into objective (aggression, regression,
withdrawal, fixation, resignation) and subjective (extrapunitive,
intropunitive, impunitive) responses (Britt and Janus 1940).
Organizational Frustration
Organizational
frustration has been defined by Paul Spector in a very similar fashion, and
refers to an interference with goal attainment or maintenance that is caused by
some stimulus condition within the organization (Spector, 1978). It has been further narrowed to be defined as the interference
with an individuals ability to carry out their day to day duties effectively (Keenan & Newton,
1984). The sources of organizational frustration
put forth by Spector include the physical environment (both natural and
man-made), the organizational structure and climate, the rules and procedures
of the organization, and individuals both in and out of the organization. In addition, the concept of situational
constraints (Peters & O'Connor,
1980)
has been hypothesized to contribute to organizational frustration (Storms & Spector,
1987). Spector (1978) suggested four reactions to
organizational frustration: 1) an
emotional response of anger and increased physiological arousal, 2) trying
alternative courses of action, 3) aggression, and 4) withdrawal. Of the behavioral reactions, only the secondfirst
one – that of trying alternative courses of action to obtain the goal – is an
adaptive response, while the other two three are maladaptive. It is likely that the emotional reaction accompanies
one of the three behavioral reactions, although the emotional reaction may be
maladaptive by itself and become a further impediment to goal attainment. Clearly, should an individual become
frustrated, it is in the best interests of the organization to have the
individual respond in an adaptive way and attempt to find another solution to
the problem in a clear decisive manner.
Spector also put forth the idea that some mild forms of frustration may
be seen as challenges rather than problems for some individuals, thus causing a
motivational effect rather than a hindering effect and increasing the
likelihood of an adaptive response rather than a maladaptive one.
Behavior exemplifying two of the threetwo maladaptive responses, in an organization, are described by Spector in his model. Examples of withdrawal behavior in an
organization could include the abandonment of a goal, absenteeism, or
turnover. Examples of organizational
aggression include interpersonal aggression, sabotage, and withholding of output. Both of these maladaptive responses are
thought to lead into a decrease in job performance. However, evidence for the
frustration-performance link is mixed, as some cases of mild frustration are
found to increase task-performance presumably due to increased arousal (Spector, 1975), whereas other studies find
that frustration actually inhibits both task performance and learning of a new
task.
Other relationships with organizational frustration have also been tested. In a sample of employed individuals, significant relationships were found between both self-reported sabotage and interpersonal aggression with level of frustration as measured by the Organization Frustration Scale (Spector, 1975). Frustration was also found to be strongly correlated to a self-reported desire to leave the place of employment. In another study of 401 employed engineers, Keenan and Newton found that organizational climate, role stress, and social support all correlated positively with environmental frustration (Keenan & Newton, 1984). Additionally, they found that frustration was significantly related to angry emotional reactions, latent hostility and job dissatisfaction.
Additional research has
shown that organizational frustration is positively correlated with several
negative behavioral reactions - aggression, sabotage, hostility and
complaining, withdrawal, and intent to quit (Storms & Spector,
1987). In an effort to examine the antecedents of
the response choice (adaptive or maladaptive) Storms & Spector also tested
for the moderating effect of locus of control, hypothesizing that individuals
with an external locus of control would exhibit more counterproductive behavior
during times of frustration than those with internal locus of control. This
hypothesis was supported, externals increased their counterproductive behavior
as frustration increased, whereas the reactions of internal stayed
constant. [I1]
Using the same Organizational Frustration scale, Jex and Gudanowski examined the role of self-efficacy beliefs and work stress (Jex & Gudanowski, 1992). They found that individual efficacy beliefs were significantly negatively correlated with level of organizational frustration, indicating that those with less belief in their abilities at their job suffered more frustration than those with high efficacy beliefs. However, they did not find that efficacy beliefs mediated the relationship between stressors and frustration, indicating that self-efficacy does not affect the level of frustration experienced due to external stressors such as situational constraints.
Situational Constraints
The concept of situational
constraints was introduced in 1980 by Peters and O’Connor in response to the
perceived hole in the human performance literature (Peters
& O'Connor, 1980). They argued that it has long been assumed
that the characteristics of the work setting play a role in performance, but it
had never been empirically tested. As
such, they introduced a framework for the study of such situational constraints
that might affect task performance, which takes into account the idea that
individuals who are otherwise capable and motivated to perform may be inhibited
by characteristics of the situation.
The situational factors that they believed to be relevant to
performance, using a critical-incidents method, were job-related information,
tools and equipment, materials and supplies, budgetary support, required
services and help from others, task preparation, time availability, and work
environment.
As such, Peters and O’Connor
hypothesized a direct link between situational constraints and performance, as
well as a direct link between situational constraints and affective reactions
such as job satisfaction or frustration.
In addition, they thought that the severity of the constraints would
affect performance differentially, in accordance with expectancy theory (Vroom,
1964). Persons who work in situations where severe
constraints are the norm may develop the belief that additional effort on their
part will not increase performance.
With regards to goals, this indicates that a long history of experience
with situational constraints inhibiting progress towards a goal (in their
model, performance) would reduce expectations of goal achievement and inhibit
productive goal-oriented action. For
example, research has found that abilities are positively related to affective
responses when situational constraints were low, but negatively related when
they were high, indicating that the level of situational constraint on the
individual affects their levels of job satisfaction and frustration (Peters,
O'Connor, & Rudolf, 1980).
Subsequent research has
examined the relationships between situational constraints and such dependent
variables as task performance, job satisfaction, frustration, turnover, and
goal commitment. In addition, several
studies reported the correlations between the negative affect caused by
situational constraints and other outcome variables such as performance, job satisfaction,
and turnover to demonstrate the direct link between affect and these variables,
as well as the indirect link to situational constraints.
In another study of 237 employed individuals in a range of
managerial and non-managerial jobs, O’Connor et al. examined the relationship
between situational constraints and the negative emotional reactions of job
satisfaction and frustration (Peters,
Chassie, Lindholm, O'Connor, & Kline, 1982). Their results indicated that the higher the
severity of overall situational constraints on the job, the greater the
reported frustration and dissatisfaction.
In addition, they also found that the average constraint score was
negatively correlated with level of effort (p<.001), motivation (p<.01)
and organizational commitment (p<.001).
Peters et al (1982) performed
another experiment to examine the link between situational constraints,
performance, and goal setting, hypothesizing that the link between goal
difficulty and goal performance would be mediated by the presence of
situational constraints. Their analysis
found that there was a direct link between situational constraints and
performance, but that goal difficulty was unrelated to
performance in the high constraint condition, but positively related to
performance in the low constraint setting.
This indicates that the effect of goal difficulty and goal setting is
dependent on the absence of constraints, showing that situational constraints
also have an indirect effect on performance through this relationship. In other words, highly motivated individuals
(those who have set difficult goals) are unable to perform at the level
anticipated in the presence of situational constraints.
Another study examined the
link between goal commitment and performance, and the impact of situational
constraints on this relationship (Klein &
Kim, 1998). They based this study on the idea that as
employees become frustrated with constraints, they cannot perform as well as
they feel they ought, and subsequently lose motivation because they no longer
expect to perform well. This also fits
in with the idea that reduced expectations would lower goal commitment. Their study of 105 salespersons found that
situational constraints was negatively related to goal commitment, indicating
that the presence of situational constraints negatively impacts the commitment
to goals.
Phillips and Freedman
performed another study of possible indirect links between situational constraints
and motivation and satisfaction, and whether or not degree of perceived
personal control over the job had any relationship (Phillips
& Freedman, 1984). They found a direct link between situational
constraints and motivation and satisfaction.
They also found that in individuals feeling a high level of personal
control, the perceived presence of high constraints actually increased the
motivation and satisfaction of individuals in jobs that were characterized by a
low motivating potential. This suggests
that individuals who perceive that they are in control of their job (internal
locus of control) will perhaps find that mild constraints increase the
challenge of an otherwise boring job.
This could be because these individuals continue to feel in control of
the outcomes of the situation, and that a constraint does not necessarily
inhibit the achievement of the goal and is seen as challenging rather than
problematic.
Locus of control was also
examined as a factor in a field study by Storms and Spector (1987), where they
found that situational constraints were positively related to perceived
frustration, and that frustration was positively related to negative behavioral
reactions. As discussed above, locus of
control was found to moderate the relationship between frustration and negative
behavior, in that individuals with an internal locus of control did not
increase their negative behaviors as a result of frustration, whereas
externally located individuals did.
Frustration and Situational Constraints
Clearly, situational
constraints can have many consequences for an employee, ranging from
frustration and dissatisfaction to negative behaviors. Situational constraints not only have a
direct effect on these negative consequences, but they also have an indirect
effect through the affective reaction of frustration. Constraints in the environment cause frustration in individuals,
and the subsequent frustration also has a direct effect on satisfaction,
motivation, performance, and the increase in negative behaviors. A meta-analysis undertaken in 1993 on the
effect of constraints and work related outcomes reports that situational
constraints have a direct effect on turnover (.21), performance (-.14),
frustration (.39), satisfaction (-.32), and commitment to the job (-.22) (Villanova
& Roman, 1993). In addition, frustration has an additional
effect on performance (-.08) and satisfaction (-.31). It would appear that both the situational constraints themselves,
as well as the affective reaction towards it (which can be moderated by such
factors as locus of control and efficacy beliefs) can influence organizational
outcomes.
Computer Anxiety
The reactions of people to computers have been studied extensively, particularly attitudes towards the computer (Loyd & Gressard, 1984; Murphy, Coover, & Owen, 1989; Nash & Moroz, 1997) computer anxiety (Cambre & Cook, 1985; Cohen & Waugh, 1989; Glass & Knight, 1988; Maurer, 1994; Raub, 1981; Torkzadeh & Angulo, 1992), and computer self-efficacy (Brosnan, 1998; Compeau & Higgins, 1995; McInerney, McInerney, & Sinclair, 1994; Meier, 1985). Each of these variables, combined with the factors listed above, can affect how frustrated individuals will become when they encounter a problem while using a computer.
The number of times a
problem has occurred before can affect their perception of the locus of
control, and therefore influence their reaction as well. This may be related to anxiety, as people with low
computer self-efficacy may be more anxious (Brosnan, 1998; Meier,
1985)
and more likely to view the computer suspiciously and react with great
frustration when something occurs, especially when they have run into it
before. Different levels of anxiety
will affect performance when something unforeseen or unknown occurs, causing
anxious people to become more anxious (Brosnan, 1998). On the other hand, the level of experience may temper this if the
prior experience increases computer self-efficacy (Gilroy & Desai,
1986)
by lowering anxiety and reducing frustration when a problem occurs. The perceived ability to fix problems on the
computer, as well as the desire to do so may also affect levels of frustration. If instead, these problems are seen as
challenges
rather than problems,, they may not be as frustrating, which is
most likely directly related to level of prior experience as well as computer
self-efficacy.
Computer Frustration
Frustration with technology is a major reason why
people cannot use computers to reach their goal, hesitate to use computers, or avoid computers
altogether. A
recent study from the Pew Internet and American Life study found that a
large percentage of people never go online, because they find the technology to be too frustrating and
overwhelming (Pew, 2003). Currently, 42% of Americans do not use the Internet, in
large part because they find it to be frustrating and confusing. This is not surprising; previous research on
user frustration found that users wastedr
nearly one-third to one-half of the time spent on the computer, due to
frustrating experiences Research on computer frustration (Bessiere, 2002;
Bessiere, Lazar, Ceaparu, Robinson, & Shneiderman, 20032).
Unfortunately,
computer applications are often designed with interfaces that are hard to use,
and features that are hard to find. Even government web sites, which are
supposed to provide easy access to government information for all citizens, are
frequently hard to use and produce high levels of user frustration (Ceaparu, 2003; Hargittai, 2003). Frustration with technology
can lead to wasted time, changed mood, and sufferedaffected interaction with colleagues.
When users in a workplace are frustrated with their computers, it can lead to
lower levels of job satisfaction (Murrell & Sprinkle, 1993). In some cases, user
frustration with technology can even lead to increased blood volume pressure
and muscle tension (Riseberg, Klein, Fernandez,
& Picard, 1998)
Research on computer frustration
has shown that that computer self-efficacy and attitudes play a
significant role in reducing the frustration levels in computing. Level of comfort with the computer and the
determination to fix a problem, which are associated with a high level of
computer self efficacy, both appear as important factors in both the immediate
experience of frustration as well as the overall frustration level after a
session of computer use.
In this study, cIn the previous study on computer frustration, computer attitude
variables mediated the experience of frustration but experience did not.
Simply using a computer, therefore, does not lessen user frustration;, rather it is
one’s attitude towards it and comfort with it.
There
is a measurable benefit to improved usability of user interfaces for lower user
frustration (Bias & Mayhew, 1994). Many well-known companies,
such as IBM, Staples, the National Football League, and Macy’s, do focus on improving their
interface design, which leads to measurable improvement ofin the bottom line (Clarke, 2001; Tedeschi, 1999). For instance, when Macy’s
made their web site search engine easier to use, the conversion rate (the rate
at which site visitors are “converted” into buyers) went up 150% (Kemp, 2001). Staples.com used feedback
from users to improve their online registration pages, to make them easier to
use. After improving the usability of the registration pages, the registration
drop-off rate (the number of people who begin registering but fail to complete
the registration) decreased by 53% (Roberts-Witt, 2001). After losing market share,
AOL yielded to customer complaints and removed a majority of the pop-up
advertisements from their service (Hu, 2002). Companies that have
re-designed interfaces for log-on screens and for user forms have seen
improvements in employee productivity that can be measured, in tens or hundreds
of thousands of dollars (Nielsen, 1994).
Having a proactive attitude towards the
computer, particularly seeing the problems with the computer as challenges
rather than problems, was shown to decrease frustration levels.
The consequences of negative
attitudes towards the computer have not been studied extensively within
organizational settings. However,
Murrell and Sprinkle examined the relationship between feelings of frustration
and confusion about the use of computers and found that these were associated
with lower job satisfaction (Murrell
& Sprinkle, 1993). This indicates that the consequences of
negative attitudes towards computers may extend to the organizational level, as the
literature on situational constraints would also suggest.
Organizational Frustration, Situational
Constraints, and the Computer
The growing
ubiquity of computers and information technologies as part of the
organizational environment suggest the increased role that equipment and tools,
and situational constraints more generally, will play in both organizational
level and individual level frustration.
The dearth of literature examining the causes and associated factors
surrounding computer frustration make it difficult to hypothesize about the
links between computer frustration and organizational frustration and other
negative consequences. However, the
literature on situational constraints gives some indications for areas of future
research. For instance, we might
speculate that the complexity of the computer as a tool leads to greater
opportunities for situational constraints to produce organizational
frustration.
Proposition 1:
The computer, as a piece of equipment and a tool
necessary to accomplish many jobs, will contribute to levels of organizational
frustration as a situational constraint.
In addition, the increasing reliance on
technical support adds another dimension to the idea of the computer as a
situational constraint – insufficient support and help in the face of computer
problems will also hinder the individuals’ progress towards their goals in the
workplace.
Proposition 2: Increasing reliance on technical support both within and without
the organization will serve as an additional situational constraint.
If the
computer becomes a situational constraint, as well as the increased reliance on
other individuals to keep this equipment running properly, all the attendant
organizational consequences of situational constraints will therefore apply to
the computer.
Proposition 3:
Frustration with computer problems will lead to
decreased job satisfaction, increased organizational frustration, a decrease in
performance due to both frustration and loss of time, a loss of motivation, and
the exhibition of maladaptive goal-attainment behaviors.
However, as
the literature on frustration indicates, particularly the literature on
computer frustration, greater computer self-efficacy will lead to a tendency to
view computer problems as challenges rather than problems, and will moderate
the relationship between computer frustration and the resultant consequences.
Proposition 4:
Greater computer self-efficacy will lead to
adaptive goal-attainment behavior and moderate the relationship between
computer frustration and the resultant consequences.
Literature
on the role of training and experience indicates that these lead to greater
computer self-efficacy, but do not have a direct link to the consequences of
negative attitudes towards the computer.
However, since training and experience increase computer self-efficacy,
and computer self-efficacy moderates this relationship, training and experience
are deemed necessary for the development of proactive attitudes.
Proposition 5:
Training and experience will increase computer
self-efficacy.
Conclusion
As the reliance upon computers as
tools in organizations becomes more prevalent, it becomes more important for
individuals to have positive attitudes towards the computer in order for them
to have adaptive reactions to the problems that they are sure to
encounter. The computer, as a piece of
equipment and a tool, is a situational constraint that has consequences for
affective reactions to the organization and job, as well as behavioral consequences
of decreased performance and possible interpersonal and organizational
aggression. As such, it is important to
look at the possible factors that could lessen these maladaptive reactions, and
find ways to lessen the impact of computer problems. One such method might be to increase training and experience of
users in order to heighten their computer self-efficacy and subsequently lower
the negative consequences. However, to
ignore the problem of the computer would be detrimental to an organization,
especially as technology becomes more complex and harder to learn. As such, it is vital that organizations give
the support to their computer users that is needed to lower the negative
attitudes and frustration, which could have devastating organizational impact.
RESEARCH
METHODOLOGY Research
Methodology
To learn more about user frustration with technology in the
workplace, data was collected through the use of modified time diaries. With a modified
time diary, usersUsers
recorded data about their
frustrations as the frustrations occuroccurred. Surveys would not be an appropriate
data collection methodology for this research, since users trying to recall
frustrations from their past experiences might over-estimate or under-estimate
the level of frustration and the time wasted (Fowler, 1993). In addition, data logging
cannot effectively measure frustration, since data logging would only work for
system errors, or other occasions when the systems indicated an error state.
There are many events that are frustrating for users (such as spam or pop-up
advertisements), and occur when the system is operating in a correct state. This same methodology was used
in the previous study of computer frustration in students (Ceaparu, Lazar, Bessiere,
Robinson, and Shneiderman, 2003).
Subjects in this experiment study were encouraged to
perform their typical work-related tasks, and record, as a part of their time
diaries, any frustrating experiences. Tasks are not pre-assigned to subjects,
because user frustration is correlated to the importance of the task (Bessière, Ceaparu,
Lazar, Robinson, & Shneiderman, 2003). When tasks are important to
users, users report higher levels of frustration than when tasks are not
important. Pre-assigned tasks would therefore not accurately model the user
frustration in an average workday. The following protocol was used:
1. Fill out demographic information (age, gender, computer experience, etc.)
2. Fill out a pre-session survey (noting current mood) (Appendix A)
3. Perform work-related computer tasks of their choosing, for a minimum of one hour total.
4. Fill out frustration experience forms, whenever the subject feels frustrated. These forms describe the cause, nature, and severity of the frustrating experience. (Appendix B)
5. Fill out a post-session survey (measuring frustration after the session ended) (Appendix C)
6. After completing the
post-session survey, subjects fill out a reimbursement form and return all of
the materials via postalsnail-mail
to the researchers.
RESULTS
Data collection took place from mid-2002 until
2003.
A
total of 50 subjects took part in the research experimenstudy. Each of these subjects was a workplace user of
computers, and each subject was
paid $25 for their participation. The workplaces represented in
this study include healthcare (15), law (3), education (8), information
technology (11), non-profit-other (5), for-profit-other (2), government (3),
and 3 subjects did not indicate their workplace. The average age of users was 35.95 years (with a
range of 23 to 76 years old). The average number of years of computer
experience was 2.38 years (with a range of less than a year, to 25 years of
experience). A total of 149 frustrating
experiences were reported, with each participant reporting between 1 and 6 experiences. Users recorded their experiences, in time diaries, for a period of 5.1 hours, on average. This paper reports the causes
and severity of the frustration, highlighting the responses to frustration, as
well as the time lost. A separate paper will address how the frustration impacted on the individuals mood and
interaction with others.
Figure 1: Demographic information
|
|
|
|
|
|
|
|
|
|
WWeb browsing and ord processingemail and e-mail produced the largest number of frustrating
experiences, probably reflecting that these applications were used most often
(Table 1). Table 1 reports
the applications that were the source of the frustrating experience. Often the frustrating experience affected the entire system, and was during
activities that participants had previously completed successfully a number of
times without error. There were
several frustrating experiences involving moving data from one application to
another type of application, such as
email content into word processing and even moving content among similar
applications, such as Word to WordPerfect. Many frustrating experiences were inhibiting but did not ultimately prevent the
task from completion.
Table One: Summary and
Demographic information from the study
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Participants
came from a wide variety of backgrounds and work environments. There were 50
participants with a total of 149 frustrating experiences. With each participant
reporting between 1 and 6 experiences. The most common response to errors was
to reboot or restart the computer, implying that the problem would fix itself
eventually.
Table
On1eTwo: Applications that were the
source of frustrating experiencesProblem Sources of reported
incidents
Problem source |
Frequency of problem sources
|
|
Web browsing |
17 |
|
|
28 |
|
Other Internet Use |
11 |
|
Video/Audio Software |
1 |
|
Word Processing |
34 |
|
Chat and Instant Messaging |
1 |
|
File Browsers |
1 |
|
Programming Tools |
2 |
|
Spreadsheet Programs |
9 |
|
Graphic Design Programs |
1 |
|
Presentation Software |
1 |
|
Database Programs |
6 |
|
Other |
37 |
|
Total |
149 |
Table 2 reports
the solutions that users took in the attempt to solve their frustrating
experiences. Most participants
were already familiar with the
frustrating experience from previous
experiences and know how to solve it (Table
2). Most solutions
involved simply redoing the task either orafter a
restarbooting and then redoing the task or twice in a row. Other solutions involved work-arounds and as a
last resort finding help externally. The type of solution taken was independent
of other demographic informationdifferences. Int is interesting to note that only
one frustrating experience did the user consult a manual, and only
in two experiences did the user consult online help.
1oneProblem
sources were based on those listed in a previous study. Often the problem affected the
entire system, and was during activities that participants had completed
successfully a number of times without error. There were several incidents
involving moving data from one application to another type of application, such
as email content into word processing and even moving content among similar
applications, such as word Word to WordPerfect. Many problems
were inhibiting but did not ultimately prevent the task from completion.
Table 23: Solutions taken by participants
Solution taken |
Frequency of solutions
|
|
I knew how to solve it because it happened before |
35 |
|
I figured out a way to fix it myself |
9 |
|
I was unable to solve it |
16 |
|
I ignored the problem or found an alternative |
20 |
|
I tried again |
5 |
|
I restarted the program |
15 |
|
I consulted online help |
2 |
|
I asked someone for help |
16 |
|
I rebooted |
29 |
|
I consulted a manual or a book |
1 |
|
Total |
148 |
TIn this study, the time lost due to frustrating experiences was one of our key measuresd. In terms of time lost, one frustrating
experience was considered to be an outlier. The one outlier frustrating experience was reported as 540 minutes to
fix the problem, and another 540 minutes to recover the problem. The cause of the frustrating experience was a
hardware problem, where the user reported that they would assign IRQs to
hardware, and every time that the computer re-booted, the operating system would re-assign those IRQs. The user reported disabling
the problem devices so that they could complete their current tasks. Due to the
large amount of time wasted, we therefore felt that this one frustrating experience should be separated out as an outlier. The user
reported two other frustrating experiences, but those were well within the
typical range reported. The other frustrating experiences from the same subject
are therefore included.. Users, in general, spend more time recovering from
an incident than initially working through the incident. Both the initial time spent on
responding to the frustrating experience, as well as the time to recover from any work lost
due to the problem, contribute to the total time lost. The method for computing percentage of time lost is as follows:
Percent Time Lost = (MS + MR) / MT
Where MS is
percentage of time lost=
(minutes spent to solve the problem, MR is) +( minutes spent to recover from
lostany work loss due to the
problem, and MT is)
---------------------------------------------------------------------------------------------------------------------------
total minutes spent on the
computer
(Ceaparu,
Lazar, Bessiere, Robinson, and Shneiderman, 2003).
For each user, the amount of
time lost to respond to the initial problem, as well as the time lost to
recover from the problem, was added for all frustrating experiences reported by that user and then divided by the
overall time spent by that user on the computer. The final figure represents
the percent of total time lost by
that user of the time that they spent on the computer. . Table
3 gives a sample of these data calculations
from one user.
Table 3. Sample of time lost statistics for one
user
|
|
Minutes |
Minutes |
Total Minutes Lost |
Total Minutes Spent on the
Computer |
Percent Time Lost t |
Percent Time |
Percent |
|
A sample user |
30 |
50 |
80 |
183 |
16.4% |
27.3% |
43.7% |
The percentages for time lost dueto solve to initial the problem, time lost to recover lost work, and total
time
lost, after being
normalized for each user, were then averaged over the population of 50 users (. The resulting averages are displayed in tTable 4). Each column in table 4 is
calculated from the data itself and not from the previous numbers in the table. Therefore, percentage time
lost is not exactly equal to the sum of the figures in the first two columns. FiguresPercentages are given for all users, and
for all users minus the one outlier frustrating experience frustrating experience (see paragraph
below).
One user reported an extremely
long duration frustrating experience, which we considered to be an outlier. The
one outlier frustrating experience was reported as 540 minutes to solve the problem, and another 540 minutes to recover
lost work. The cause of the frustrating experience was a hardware problem,
where the user reported assigning IRQs to hardware, and every time that the computer
re-booted, the operating system would re-assign those IRQs. The user reported
disabling the problem devices to complete the current task. Due to the large amount
of time wasted, we therefore felt that this one frustrating experience should
be separated out as an outlier. The user reported two other frustrating
experiences, but those were well within the typical range reported. The other
frustrating experiences from the same subject are therefore included (Table 4).
Most Participants were already familiar with
problems from previous experience. Most solutions involved simply redoing the
task either after a restart or twice in a row. Other solutions involved work
arounds and as a last resort finding help externally. The type of solution
taken was independent of other demographic information.
Table a4: Breakdown
of time spent and lost by participants
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Table 3b: Breakdown of time spent and lost by participants (with
Outliers)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Table 43c.: NEW Breakdown of averageAverages for percent time lost due
to initialsolve problem, average time to recover lost work, and average total time
lost for all users time spent and lost by
participants
|
|
Percent Time Lost to Solve Problem |
Percent Time to Recover Lost
Work |
Percent |
|
Total w |
20.3 |
20.2 |
42.7 |
|
Total w |
21.5% |
22.2% |
43.7 |
Table 3d: NEW Breakdown of time spent and lost by
participants (with Outliers)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Table 3e: Time Statistics
|
|
|
|
|
|
|
|
Table 5 reports the total minutes
lost by application source. Problems
with word processing cost participants the most amount of time in total (1225 minutes lost), followed by problems with email (666 minutes lost) (Table 5). This did correlate towith the applications that caused the highest number of
frustrating experiences. The applications that were uncommon sources of frustrating experiences (such as programming tools, database software, and presentation software) often required more time per
incident, as the problems were rare and
complex to solve. Tables 3a-d
represent important statistics with and without the outliers of the study. A and B were
calculated by first adding up the times of all participants and then calculating percentages. C and D were calculated by finding the
percentages for each participant and then adding up and averaging those
results. Table 3e lists some other
miscellaneous statistics.
Time lost was divided by the time to fix the
problem and the time required to get back up to speed. Participants spent more
time recovering from an incident than initially working through the incident.
This includes recovering lost work and the time to reaccomplish the initial
task. Time spent was based on start and stop times recorded by the user. The
percentage is the amount of time that could have been utilized, had the
incident not occurred.[ADAM: WE NEED TO SHOW TWO
DIFFERENT ANALYSES HERE: WITH THE OUTLIER, AND WITHOUT THE OUTLIER]
Table
55:
Breakdown of total minutes lost and average time minutes lost per frustrating experienceincident
by application problem
source
|
Problem source |
Reports |
||
|
|
Total minutes |
# |
Average |
|
|
666 |
28 |
23.8 |
|
Web Browsing |
244 |
17 |
14.4 |
|
Other Internet Use |
105 |
11 |
9.5 |
|
Word Processing |
1225 |
34 |
36.0 |
|
File Browsers |
4 |
1 |
4.0 |
|
Video/Audio Software |
20 |
1 |
20.0 |
|
Programming Tools |
140 |
2 |
70.0 |
|
Graphic Design Programs |
5 |
1 |
5.0 |
|
Database Programs |
335 |
6 |
55.9 |
|
Chat and Instant Messaging |
2 |
1 |
2.0 |
|
Presentation Software |
105 |
1 |
105.0 |
|
Spreadsheet Programs |
604 |
9 |
67.1 |
|
Other |
865 |
37 |
23.4 |
Total |
4320 |
149 |
28.9 |
Note: FE=number of frustrating experiences
Table 6 lists the specific causes
of the frustrating experiences. Problems with word processing
cost participants the most amount of time, followed by problems with email.
This did correlate to the two most common problem sources. The uncommon sources
often required more time per incident as the problems were rare and complex to
solve. In some cases minutes lost was also upinfluenced by to external factors such as help desks and tech
support. More often than not time would not have been saved with a backup copy
of the data taken just before the error and restoring afterwards. Categories of frustrating experiences were based loosely
on a previous study (Ceaparu, Lazar, Bessiere, Robinson, and
Shneiderman, 2003)
with minor changes to accommodate terms used by our participants (Table 6). Major categories were
grouped by the behavior described in each frustrating experience. System crashes were the most
commonly-reported frustrating experience, accounting for 21 of the 149, and were caused by specific
programs as well as the operating system itself. While many of these problems are hardware-related
or technical-related (such as printing problems and system freezescrashes), there were a number of
frustrating experiences that were caused by interface-related issues (such as
uncontrollable pop-up windows, hard to find features, and unpredictable behavior
of application, and unclear error messages). For instance, there were 19 experiences with
missing/hard-to-find/unusable features, 4 experiences with uncontrollable pop-up
windows, and 5 experiences with unclear error messages. These interface-related causes
of user frustration are easily solvable, when attention is paid to appropriate user
interface design.
As discussed in previous portions of the paper, when these interface
improvements are made, it leads to improved user productivity and organizational profitability.
Table
66: Specific causes of frustrating experiencesBreakdown of
5 main categories and many subcategories of problems
Internet |
Applications |
Operating System |
Hardware |
Other |
|
{ 7 } |
Missing / Hard to find Unusable
Features { 19 }
|
System { 21 }
|
Printing Problems { 10 } |
User Kicked from System { 5 }
|
|
Sending / Receiving Email and
accessing attachments { 6 }
|
{ 11 } |
File Browser Operations { 2 }
|
Hardware Conflicts { 3 } |
Multi user File Access and Permission Issues { 5 }
|
|
Uncontrollable Pop-up window { 4 }
|
Buggy, Incorrect behavior of program { 10 }
|
Multitasking Failure { 1 } |
Device Failures { 3 } |
Password Not working { 2 }
|
|
{ 3 } |
Excessive Slow
Operation { 8 } |
|
|
Virus /
Malicious Program { 1 }
|
|
Browser Failure { 2 }
|
Unpredictable Response of program { 6 }
|
|
|
Local Network Connection Access
Failure { 1 }
|
|
File Download Failures { 2 } |
Unclear Error Messages { 5 }
|
|
|
Power Failure { 1 }
|
|
Plug-in Failure { 1 }
|
Installation Issues { 4 }
|
|
|
|
|
|
Application Crash that Froze the
entire System { 4 }
|
|
|
|
|
|
{ 1 } |
|
|
|
Note: some
responses were left blank by participant
The participants expresses strong eTables 7 and 8 display the data on the emotional reactions to the frustrating experiences (. Table
7) reports on how users felt after
the frustrating experiences. For
instance, in 60 of the frustrating experiences, users felt angry at the
computer, in 34 experiences, users felt
helpless/resigned, and in 15
experiences, users felt angry at
themselves. It is important to
note thatS since
users may have more than one emotional reaction, these numbers for table 7 will add
up to more than the 149 frustrating experiences reported. Table 8 reports
the level of frustration for each of the frustrating experiences. It is important to note thanUsing a 1 to 9 numeric scale, 106/149 of the frustrating
experiences were reported to have frustration levels of 7, 8, or 9 , very high frustration levels, indeed(. Figure
1) displays the same frustration levels as table 8. These high
levels of frustration can have an impact on the
human bodyphysiological variables. For instance, in a previous
study of user frustration, it wasresearchers found that when typical users
get frustrated with their computer, it affects blood volume pressure (Riseberg, Klein, Fernandez,
& Picard, 1998).Categories were based loosely
on a previous study. Major categories were grouped by the behavior described in
each incident. System crashes were the most commonly frustrating, and were
caused by specific programs as well as the operating system itself. Most of
these problems were non repeatable and subcategories involved more than one
participant. For example all of the pop-up window errors did not belong to one
person.
Table 76: User feelings per incident
|
Expressed Feeling |
Number of Reports |
|
Angry at the computer |
60 |
|
Angry at yourself |
15 |
|
Helpless / Resigned |
34 |
|
Determined to fix it |
27 |
|
Neutral |
17 |
|
Other |
26 |
Note:
Some
participants had multiple feelings per incident
Table 8767: Number of incidents for each
level of frustration
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Note: some responses were left blank by participant
Figure 17: Bar graph of reported
frustration levels
[ADAM: WE
SHOULD ALSO HAVE A BAR GRAPH OF THIS CHART]???
Unfortunately,
it seems that these frustrating experiences are not rare occurrences that happen on an infrequent basis.
From the
frustrating experiences reported, users were asked to indicate whether this
same event had occurred previously, and if so, how often (. Table
8)9 displays their responses related to the frequency
of the problem.. For instance, for 25 of the frustrating
experiences, users reported that the same event occurs more than once a day.
For 21 of the frustrating experience, users reported that the event occurred several times a week. It is clear that users repeatedly face problems,
and these problems interrupt their day.
Most users reported high levels of frustration, and
some level of effect on their coworkers. The more frequent problems and the
ones that took the most time were associated with higher levels of frustration.
Unimportant tasks had lower levels of frustration.
[ADAM: WE NEED
THE DATA ON HOW OFTEN THESE INCIDENTS HAPPEN]
Table 898:
Frequency of problems
|
Frequency
of problem |
Number
of reports |
|
7 |
|
|
15 |
|
|
11 |
|
|
21 |
|
|
29 |
|
|
9 |
|
|
29 |
Note:
Some responses left blank by participant
Discussion
From the results of this
study, it is clear that user frustration is a problem in the workplace. Since uUsers lose largemore than 40% quantities of their time,
and these frustrationg experiences have an impact on both
the
individuals, and their time lost has an impact on the organizations. From an individual point of view, users waste a large amount of time, which impacts
onslows their completion of work,
limiting their time with family, friends, and co-workers. It also can affect
their emotional state. TAnd these frustrating experiences also impactharm on
the organizations which these individuals work for. Just think about how much more
profitable a company would be if more of that time was productive. And this
isn’t the workers slacking off or wasting their time—this is the users that are
trying to complete their tasks, but are facing frustrating experiences, leading
them to take more time on their tasks. The time wasted due to poor computer
interface design is staggering. The time wasted
has a large monetary value to organizations. by undermining productivity,
lowering quality, and raising stress levels. The moneyinvestment in spent to improvinge
user interfaces iswould yield large payoffs; several studies suggest more than justified by the
monetary value of time saved. In some cases, the that the cost of the interface improvement is made up 5, 10,
or 20 times over (Bias and Mayhew, 1994).
It is clear from this study and previous work that user
frustration is a major problem. In the previous study of 111 subjects, the subjects were university
students, and their friends and family members. This current study was limited to workplace
users. However, the results of the two studies were surprisingly similar.
Next Section: A Comparison of
the Workplace data and the student data
Table 9 reports
the The top 3 problem applications causing a
frustrating experiencesources from both the previous study
and the current study. The applications causing the most frequent
frustrating experiences for the student frustration studystudy were web browsing, email and
word processing. In the workplace frustration study, the top 3 application problems sources
encountered by the users were the same, but in reverse order: word processing,
email, web browsing.
The top 3 solutions taken
by the participants to solve the problems that occurred in the student study were:
for the
student study - they knew how to solve it from previous
experience, they figured out a way or they were unable to solve it. For the workplace study, the
top 3 solutions cited were; for the workplace study - that they knew how to solve it from previous experience, they rebooted, or ignored the
problem/found an alternative.
Table
9: Applications causing the largest
numbers of frustrating experiences
|
Student
Study-Causes of
Frustration |
Workplace
Study-Causes of Frustration |
|
1. Web Browsing |
1. Word Processing |
|
2. E-mail |
2. E-mail |
|
3. Word Processing |
3. Web Browsing |
In both studies, the levels of frustration wereis at the high
end of the scale. For
instance, 7,8, and 9 are the highest scores on the frustration scale, and in
both studies, large percentages of subjects reported their frustrations being in that
range. Levels 7, 8 and 9 of
frustration are the top 3 levels for both the student study (7: 68, 8: 77, 9:
91) In the
student frustration study, 63.3% (236 out of 373) of the frustrating experiences caused high levels
of frustration. In the workplace study, 71.1% (106 out of 149) of frustrating experiences caused high levels of
frustration. and the workplace study (7:23, 8:33, 9:50).
The amount of
time lost was also similar in the different frustration studies. Table 10 describes the time lost in the various studies, both
with and without outliers. The numbers are very similar. In the previous study
with students, there were two phases: self-reports and observations. These data
points are listed separately. In the previous study with students, the average percentage of time lost ranged from 38.9% (for self-reports
without the 5 outliers) to 50.1% (for
self-reports with the
5 outliers). In this workplace study, the
average percentage of time lost ranged from 42.7% (without outliers) to 43.7% (with outliers).
We think that the difference in ranges was logical, due to the numbers of
subjects involved in the two studies. In the previous student study, 111
subjects took part, whereas in this workplace study, only 50 users took part. With a larger number of users
taking part, it logically follows that there will be more outliers, and
therefore, a wider percentage spread.
Table 10109: Time lost in the variousthree studies, with and without
outliers.
|
|
Average
time lost (with outliers) |
Average time lost |
|
Student study self-reports |
50.1% |
38.9% |
|
Student study observations |
49.9% |
41.9% |
|
Workplace study |
43.7% |
42.7% |
From the analysis in the student study, we get the
following results in terms of time lost: the average time lost per individual
for UMD reports was 47.8% (37.9% without the outliers) and for Towson 53.1%
(43.5% without the outliers); the average time lost of 50.1% (38.9% without the
outliers) from self-reports and 49.9% (41.9% without the outliers) from
observations.
From the
analysis in the workplace study, the average time lost per individual is 28.1%.
When looking at the specific causes of the frustrating experiences categories
and subcategories of problems that occurred, the student study
finds that the top 5 were: error messages, timed out/dropped/refused
connections, application freezes, missing/hard to find/unusable features, long
download time. The workplace study finds that the top 5 were: OS crashes,
missing/hard to find/unusable features, application crashes, hardware problems,
buggy/undesirable behavior of program. Many of these problems (such as error messages,
hard-to-find features, and undesirable behavior) are caused by poorly-designed
interfaces, and therefore, can be solved with more usability testing and more
user involvement in the interface development. It is interesting to note that some of the causes
of frustration for the student participants, such as timed out/dropped
connections, and long download times, did not appear as frequent frustrations
for workplace participants. It is likely that the network connection at a
workplace is of higher quality and speed, therefore less likely to cause
frustration due to
either response time or dropped connection. However, many of the software applications are the same, regardless of user population
or location, and are highly likely to cause frustration.
It is clearThis study with 50 workplace users adds to the growing evidence that user frustration is a
major problem. Further
studies with a narrower focus may isolate and measure contributing factors, but there is enough
evidence to encourage change in the industry.
This study, in
a new population of workplace users, helped to confirm the problem of user
frustration. What can be done about this problem? BadImproving user interface design,
causing user frustration, is a solvable problem is one clear opportunity because
the payoffs will be immediate and benefit many users. To build better interfaces,
more user involvement is needed in the interface design process. Designers should follow the
interface guidelines that exist. AndUser tra users needs to be provided with
more training and
documentationwill also help, especially if it addresses problem
solving strategies that will help build self-efficacy. Even small changes in the interface can make
a big impact on user satisfaction. For instance, in recent studies of the FedStats web
site, changing the interface of a governmental web site increased user
satisfaction and performance nearly 100% (Ceaparu and Shneiderman, 2004). But these frustrations can actually be “engineered-out” of the system. And many of
these frustrations are easily solvable. For instance,
it is noted in the data thatSincereported beingare , interface designers should be directed to review
all messages and instructions. The cause of these specific frustrations are generally unclear wordIf link
titles are unclear for users,
or are not where the users expect, they may not be able to complete their tasks (Daniel and Lazar, 2004). Unclear
wording has been found as a major problem in interface design, regardless of
the user population or the task. For
instance, in a usability study of a university web site, 5 users all failed to
find the information that they were looking for (current course schedules)
because the information was listed under an unclear heading (“Student
Life”). From a technology coding point
of view, changing the words displayed is
relatively simple. In
addition, the usability methods needed to find out that the wording
is unclear, are also relatively simple. Paper-based
usability testing methods such
as card sorting or paper prototypes can help find flaws in interface wording. Since many users reported being frustrated by
unclear error messages or by hard-to-find features, interface designers should
be directed to review all messages and instructions. Good guidelines for error message design have
existed since 1982, but these guidelines are rarely followed. (Shneiderman,
1982). Error messages should be positive, provide information for users (in
their language) on what occurred, and offer suggestions on how to continue. Current error messages rarely assist users (see
figure 2 for an example of this). Improved
error messages can reduce user frustration while making users more satisfied
and productive. (Lazar and Huang, 2003). While
all causes of user frustration are not as easily solvable, a large percentage
of user frustrations ARE solvable. And
there are many resources out there to help improve interface design, such as
books, automated software tools, guidelines, and other resources (See www.hcibib.org
or www.hcirn.com
for more information).
Figure 2. An unclear error
message
The iImplications
for sDifferent Stakeholders
might be separated out by:
Designers -–
As
designers, utilizing the results of studies such as this can yield the most benefits of all. B : can build more productive
systems by learning what frustrates users in
the workplace. designers can build more productive systems, that
is systems with less frustration. Systems can be modified
not only to have fewer errors but also to be more helpful. This may include better error messages,
better and helpful descriptions of problems which can reduce the time needed to
fix an issue, as well as designs based more closely on the way users work particularly with respect to how end users
handle errors. This would improve efficiency overall as systems would be better
equipped to handle problems faster and allow for the system
to get back to operating normally (without problems) and in general make things more usable.
Managers -–
Managers can benefits
but
by learning where
frustrations occur within computing systems of their employees. This would let them see the bigger picture of how to improve
processes as well as how to deal with frustrations in the workplace by
understanding them. One such example might be allowing the manager to make
better decisions in deploying systems and choosing the right IT support for
employees. In other words less frustrating systems yield ahelp them to construct a more productive workplace, reduce
workflow bottlenecks, and can make the
environment more manageableproduce more satisfied employees in general. They can recommend training for employees and make
more appropriate choices in software acquisitions.Managerial choices involving systems with varying
frustration levels can only be made when understanding the causes and
implications of IT related workplace frustrations.
Users -–
End Users will ultimately benefit from the
improvements made to systems based on this and similar
studies. This study also gives users a method
for detailing and explaining errors, rather than simply working around them or
ignoring them. Without such interaction and insight into where users get
frustrated, improving things and reducing the frustration in the workplace is more difficult.of computers will appreciate
learning that they are not alone in their frustrations. They can take steps to improve their training and
increase their knowledge,
but they can accelerate improvement by being consumer activists who report
problems, complain to designers, and suggest improvements.
Policy
makers, like managers, benefits
from the larger picture and by seeing where inefficiencies occur. It is highly possible to improve these frustrations
through policy. Policies could be adapted to provide better solutions for
handling incidents. Of particular
importance would be policies and procedures for users interacting with help
desk and IT support (which was a source of frustration in this study). By knowing where bottlenecks occur, policy can be modified to give quicker solutions and to make
things easier not only for users, but for a company’s IT support and infrastructure as well.
ITInformation
Technology
Staff -–
IT staff can be better prepared to handle
frustrated users and learn which type of technical problems related to which levels ofproduce the largest frustration. This can help things
move more smoothly and even help IT staff make better recommendations to managers and policymakers. IT Staff should also
be better situated as
the middleman and be able
to get better information between
users and vendors when
understanding frustration in the workplace.
Policymakers, in industry and government, should recognize the
severity of the productivity loss due to user frustration. Increased research
funding, improved training, better data collection, and increased public awareness of the problems will help produce appropriate changes.
Essentially one ideal situation
in which the implication might be turned into to practices is this. A
frustrated user is dealing with a system problem. They contact the IT staff, explain the source of the
frustration. The IT staff tries to help as best they can, then relays a summary of the
incident to the manager, who then evaluates the
frustration and talks to the policy makers. Together they improve the process of handling
such incidents by making small adjustments. In addition they are better
informed for future decisions that involve potentially frustrating systems.
This results in filtering back down to
the user, who now has an easier time when a frustrating incident occurs and can
be more productive and recover quicker. IT Staff can then do their
job more efficiently and give better information to the managers… This repeatedly refined process raises the bar with each iteration
and makes a workplace better as a whole and is based on understanding the frustrating
incidents themselves (and this understanding results from studies such as this).
Summary
Acknowledgements
The lead author of this
article was partially supported by Training Grant No. T42/CCT310419 from the
Centers for Disease Control and Prevention/National Institute for Occupational
Safety and Health. The contents are solely the responsibility of the
author and do not necessarily represent the official views of the National
Institute for Occupational Safety and Health.
We appreciate partial support from National Science Foundation grant for
Information Technology Research (#0086143) Understanding the Social Impact of
the Internet: A Multifaceted
Multidisciplinary Approach
and National Science Foundation grant for the Digital
Government Initiative (EIA 0129978): Towards a Statistical Knowledge Network.
We acknowledge the assistance
of Deborah Carstens and Robert Hammell, who both provided comments on an
earlier draft of this paper.
REFERENCES
Bandura, A. (1973). Aggression: A Social Learning
Analysis. Englewood Cliffs, NJ: Prentice-Hall.
Bandura, A. (1986). Social Foundation of Thought and
Action: A Social-Cognitive Theory.
Englewood Cliffs, NJ: Prentice Hall.
Bandura, A. (1997a).
Self-Efficacy. Harvard
Mental Health Letter, 13(9), 4-6.
Bandura, A. (1997b). Self-Efficacy: The Exercise of Control.
New York: Freeman.
Barker, R. (1938). The Effect of
Frustration upon the Cognitive Ability. Character
and Personality, 7, 145-150.
Barker, R., Dembo, T., &
Lewin, K. (1965). Frustration and Regression: An Experiment with Young
Children. In R. Lawson
(Ed.), Frustration: The Development of a
Scientific Concept. New York: MacMillan Publishing
Co.
Baron, R. A. (1977). Human Aggression.
New York, NY: Plenum.
Berkowitz, L. (1978). Whatever
happened to the frustration aggression hypothesis? American Behavioral Scientist, 21(5),
691-708.
Bessiere, K. (2002). COMPUTER FRUSTRATION: A TEST OF A
SOCIAL-PSYCHOLOGICAL MODEL. Unpublished Masters Thesis,
University of Maryland, College Park, College Park, MD.
Bessiere, K., Lazar, J., Ceaparu,
I., Robinson, J., &
Shneiderman, B. (2002). Social Psychological Factors of End User Frustration
and Their Implications. forthcoming.
Bias, R., & Mayhew, D. (Eds.). (1994). Cost-Justfying Usability.
San Francisco: Morgan Kaufmann Publishers.
Britt, S. H., & Janus, S. Q.
(1940). Criteria of
Frustration. The
Psychological Review, 47(6), 451-469.
Brosnan, M. J. (1998). The Impact
of computer anxiety and self-efficacy upon performance. Journal of Computer Assisted
Learning, 14, 223-234.
Cambre, M. A., & Cook, D. L.
(1985). Computer Anxiety:
Definition, Measurement, and Correlates. Journal
of Educational Computing Research, 1(1),
37-54.
Campion, M., & Lord, R.
(1982). A Control Systems Conceptualization of the Goal-Setting and Changing
Process. Organizational
Behavior and Human Performance,
30, 265-287.
Ceaparu, I., Lazar, J., Bessiere,
K., Robinson, J., & Shneiderman, B. (20023,
in press). Determining Causes and Severity
of End-User Frustration. International
Journal of Human-Computer Interactionforthcoming.
Ceaparu,
I., and Shneiderman, S.
(2004,
in press). Finding Governmental Statistical Data on the Web:
Three Empirical Studies of the FedStats Topics Page.
Journal of
the American Society for Information Science and Technology.
Clarke, J. (2001). Key
factors in developing a positive
user experience for children on the web: A case study.
Proceedings of the Human Factors and the Web 2001, Available at: http://www.optavia.com/hfweb/index.htm
Cohen, B. A., & Waugh, G. W.
(1989). Assessing Computer
Anxiety. Psychological
Reports, 65(1), 735-738.
Compeau, D. R., & Higgins, C.
A. (1995). Computer Self-Efficacy: Development of a Measure and Initial Test. MIS Quarterly, 19(2),
189-211.
Daniel,
J., and Lazar, J. (2004). Improving the Usability
of Governmental Statistics Presented on the Web. Working
paper.
Dollard, J., Doob, L. W., Miller,
N. E., Mowrer, O. H., & Sears, R. R. (1939). Frustration and Aggression.
New Haven: Yale University Press.
Ferster, C. B. (1957). The
Function of Aggression and
the Regulation of Aggressive Drive. The
Psychological Review, 71, 257-272.
Fowler, F. (1993). Survey
Research Methods (2nd ed.). Newbury Park,
California: Sage Publications.
Freud, S. (1921). Types of Onset
and Neurosis. In J. Strachey (Ed.), The
Standard Edition of the Complete Psychological Works of Sigmund Freud
(Vol. 12, pp. 227-230). London: Hogarth Press.
Gilroy, F., & Desai, H.
(1986). Computer Anxiety: Sex, Race, and Age. International Journal of
Man-Machine Studies, 25(1), 711-719.
Glass, C. R., & Knight, L. A. (1988). Cognitive Factors
in Computer Anxiety. Cognitive
Therapy and Research, 12(4), 351-366.
Hu, J. (2002). AOL's Pop-Up Sacrifice. CNET News,
pp. Available at: http://news.com.com/2100-1023-962345.html.
Jex, S. M., & Gudanowski, D.
M. (1992). Efficacy Beliefs and Work Stress: An Exploratory Study. Journal of Organizational
Behavior, 13, 509-517.
Keenan, A., & Newton, T. J.
(1984). Frustration in Organizations: Relationships to Role Stress, Climate, and Psychological Strain. Journal of Occupational
Psychology, 57, 57-65.
Kemp, T. (2001, November 28, 2001). Macy's Doubles
Conversion Rate.
InternetWeek.com, pp. Available at: http://www.internetwk.com/story/INW20011128S20010004.
Lazar,
J. and Huang, Y. (2003). "Improved Error Messages for Web Browsing: An
Exploratory Experiment". In J. Ratner (ed.) Human Factors and Web
Development, 2nd Edition, Mahwah, NJ:
Lawrence Erlbaum Associates, 167-182.
Klein, H. J., & Kim, J. S.
(1998). A Field Study of the Influence of Situational Constraints,
Leader-Member Exchange, and Goal Commitment on Performance. Acadamy of Management Journal, 41(1), 88-95.
Lincecum, L. (2000). The Effects of Software Disruption
on Goal Commitment, Task Self-Efficacy, Computer Self-Efficacy, and Test
Performance in a Computer-Based Instructional Task.
Unpublished Doctoral Dissertation, Texas Tech University.
Locke, E. A. (1996). Motivation Through Conscious Goal Setting. Applied Preventative Psychology, 5,
117-124.
Locke, E. A., & Latham, G. P.
(1990). A
Theory of Goal Setting and Task Performance.
Englewood Cliffs, NJ: Prentice Hall.
Locke, E. A., & Latham, G. P.
(2002). Building
a Practically Useful Theory
of Goal Setting and Task Motivation: A 35-Year Odyssey.Unpublished
manuscript.
Loyd, B. H., & Gressard, C.
(1984). The Effects of Sex, Age, and Computer Experience on Computer Attitudes.
AEDS Journal, 18(2),
67-77.
Maurer, M. M. (1994). Computer Anxiety Correlates and What
They Tell Us. Computers
in Human Behavior, 10(3), 369-376.
McInerney, V., McInerney, D. M.,
& Sinclair, K. E. (1994). Student Teachers, Computer Anxiety, and Computer
Experience. Journal
of Educational Computing Research, 11(1),
177-189.
Meier, S. (1985). Computer
Aversion. Computers
in Human Behavior, 1(1), 171-179.
Mowrer, O. H. (1938a). Preparatory
Set (Expectancy) -- a Determinant in Motivation and Learning. Psychological Review, 45,
62-91.
Mowrer, O. H. (1938b). Some Research Implications of the
Frustration Concept as Related to Social and Educational Problems. Character and Personality, 7,
129-135.
Murphy, C., Coover, D., &
Owen, S. (1989). Development and Validation of the Computer Self-Efficacy
Scale. Educational and Psychological Measurement, 49,
893-899.
Murrell, A. J., & Sprinkle, J.
(1993). The Impact of Negative Attitudes Toward Computers on Employees'
Satisfaction and Commitment Within a Small Company. Computers in Human Behavior, 9,
57-63.
Nash, J. B., & Moroz, P. A. (1997). An Examination of
the Factor Structures of the Computer Attitude Scale. Journal of Educational Computing
Research, 17(4), 341-356.
Nielsen, J. (1994). Usability Engineering. Boston:
Academic Press.
NTIA. (2001). A Nation Online: How Americans are Expanding Their Use of the
Internet. Washington, DC: National
Telecommunications and Information Administration.
Peters, L. H., Chassie, M. B.,
Lindholm, H. R., O'Connor, E. J., & Kline, C. R. (1982). The Joint
Influence of Situational Constraints and
Goal Setting on Performance and Affective Outcomes. Journal of Management, 8(2), 7-20.
Peters, L. H., & O'Connor, E.
J. (1980). Situational Constraints and Work Outcomes: The Influences of a
Frequently Overlooked Construct. Acadamy
of Management Review, 5(3),
391-397.
Pew Internet and American Life
Project. (2003). The Ever-Shifting Internet Population:
A new
look at Internet access and the digital divide. Report available at: http://www.pewinternet.org/reports/toc.asp?Report=88 and
downloaded on April 17, 2003
Peters, L. H., O'Connor, E. J.,
& Rudolf, C. J. (1980). The Behavioral and Affective Consequences of
Performance-Relevant Situational Variables. Organizational Behavior and Human
Performance, 25(79-96).
Phillips, J. S., & Freedman,
S. M. (1984). Situational
Performance Constraints and Task Characteristics: Their Relationship to
Motivation and Satisfaction. Journal of Management, 10(3), 321-331.
Raub, A. (1981). Correlates of Computer Anxiety in
College Students. Unpublished Doctoral
Dissertation, University of
Pennsylvania.
Riseberg, J., Klein, J., Fernandez, R., & Picard, R.
(1998). Frustrating
the user on purpose: Using biosignals in a pilot study to detect the user's
emotional state. Proceedings of the CHI 1998: ACM
Conference on Human Factors in
Computing Systems, 227-228
Roberts-Witt, S. (2001, September 25, 2001). A Singular
Focus. PC Magazine.
Shneiderman,
B. (1982). System message design: Guidelines and experimental results, In Directions in Human-Computer
Interaction, Badre, A. and Shneiderman, B. (Editors), Ablex Publishing
Company: Norwood, NJ, 55-78.
Shorkey, C. T., & Crocker, S.
B. (1981). Frustration theory: a source of unifying concepts for generalist
practice. Social
Work, 26(5), 374-379.
Spector, P. E. (1975).
Relationships of Organizational
Frustration with Reported Behavioral Reactions of Employees. Journal of Applied Psychology, 60(5),
635-637.
Spector, P. E. (1978).
Organizational Frustration: A Model and Review of the Literature. Personnel Psychology, 31(815-828).
Storms, P. L., & Spector, P. E. (1987).
Relationships of Organizational Frustration with Reported Behavioural
Reactions: The Moderating Effect of Locus of Control. Journal of Occupational
Psychology, 60, 227-234.
Tedeschi,
B. (1999, August 30, 1999). Good Web Site Design Can Lead to Healthy Sales. The New York Times
Torkzadeh, G., & Angulo, I. E.
(1992). The Concepts and Correlates of Computer Anxiety. Behavior and Information
Technology, 11(1), 99-108.
Villanova, P., & Roman, M. A.
(1993). A Meta-Analytic Review of Situational
Constraints and Work-Related Outcomes: Alternative Approaches to
Conceptualization. Human Resource Management Review, 3(2), 147-175.
Vroom, V. (1964). Work and Motivation. New York: Wiley.
Pre-Session Survey (Appendix A)
Email address: __________________________________
Section I: Demographic
Information
1. Age __________
2. Gender: F M
3.
Education:
|
___ High School Graduate |
___ Fresh/Soph in College |
___ Jr./Sr. in College |
|
___ College Graduate |
___ Masters Degree |
___ Doctoral-level |
4.
In
what field are you employed? ______________________________
5.
What
is your job title?
________________________________
Section II: Computer
Experience and Attitudes
1.
How many years have you been
using a desktop or laptop computer for home or work use? _______
2.
How many hours per week do you
use a desktop or laptop computer?
______
3.
What type of Operating System
is installed on the computer that you are currently using?
|
___ DOS |
___ MacOS |
___ Unix/Linux |
___ Windows 95 |
|
___ Windows NT |
___ Windows 98 |
___ Windows ME |
___ Windows 2000 |
|
___ Windows XP |
|
|
|
4.
What type of applications and
programs do you typically use? (check all that apply)
|
___ Email |
___ Other Internet Use |
___ Graphic Design Programs |
|
___ Chat/Instant Messaging |
___ Word Processing |
___ Programming Tools |
|
___ Web Browsing |
___ Spreadsheet Program (Excel) |
___ Database management/ Searching |
|
___ Presentation Tools (powerpoint) |
___ Other (please explain) ___________________________ |
___ Multimedia (audio/video) |
5.
How many years have you been
using the internet? ___________
6.
How many hours per week do you
spend online? Please indicate the amount of time that you are actually using
the computer while online, not simply the amount of time you are connected to
the internet. _________
7.
At work, do you ___ Have a permanent connection to the
internet OR ___ dial in through a modem
8.
Which of the following do you
do when encountering a problem on the computer or application that you are
using?
|
___ try to fix it on my own |
___ consult a manual/help
tutorial |
___ Ask help desk/ consultant for help |
|
___ Ask a friend/relative
for help |
___ Give up or leave it
unsolved |
|
9.
How sufficient is your
computer software and/or hardware for the work that you need to do?
Not at All
1 2 3 4 5
6 7 8 9 Very
Sufficient
Section III: For the following
questions, please choose the number that best corresponds to your feelings
|
1.
Computers make me feel:
2. When you run into a problem on the computer or
an application you are using, do you feel:
3. When you encounter a problem on the computer
or an application you are using, how do you feel about your ability to fix it?
4. How experienced do you think you are when it
comes to using a computer?
5. When there is a problem with a computer that I
can't immediately solve, I would stick with it until I have the answer.
6. If a problem is left unresolved on a computer,
I would continue to think about it afterward.
7. Right now, how satisfied with your life are
you?
8. How often do you get upset over things?
9. Right now, my mood is:
|
FRUSTRATING EXPERIENCE FORM—(Appendix B)
Please fill out
this form for each frustrating experience that you encounter while using your
computer during the reporting session.
This should include both major problems such as computer or application
crashes, and minor issues such as a program not responding the way that you
need it to. Anything that frustrates
you should be recorded.
1.
What
were you trying to do?
2.
On
a scale of 1 (not very important) to 9 (very important), how important was this
task to you?
Not very important 1
2 3 4
5 6 7 8 9
Very Important
3.
What
software or program did the problem occur in? If the problem was the computer
system, please check the program that you were using when it occurred (check
all that apply).
|
___ ___
email |
_______ file browsers |
___ ____presentation software (e.g.
(e.g. powerpoint) |
|
___ ___
chat and instant messaging |
___ ___
spreadsheet programs (e.g.
(e.g. excel) |
mult____media
(audio/video software) |
|
___ ___
web browsing |
___ ___
graphic design |
________other
__________________ |
|
_______ other internet use |
___ ___
programming tools |
|
|
)))_____ word processing |
_______ database management/ s searching software |
|
4.
Please
write a brief description of the experience:
5.
How
did you ultimately solve this problem? (please check only one)
|
___ I knew how to solve it because it has
happened before |
___ I ignored the problem or found an
alternative solution |
|
___ I figured out a way to fix it myself
without help |
___ I was unable to solve it |
|
___ I asked someone for help. Number of people asked ___ |
____I tried again |
|
___ I consulted online help or the
system/application tutorial |
____I restarted the program |
|
___ I consulted a manual or book |
|
|
___ I rebooted |
|
6.
Please
provide a short step by step description of all the different things you tried
in order to resolve this incident.
7.
How
often does this problem happen? (please check only one)
___ more than once
a day ___ one time a day ___ several times a week ___ once a
week
___ several times
a month ___ once a month ___ several times a year ___ first time it happened
8.
On
a scale of 1 (not very frustrating) to 9 (very frustrating), how frustrating
was this problem for you?
Not very frustrating 1 2 3
4 5 6 7 8
9 Very frustrating
9.
Of
the following, did you feel:
___ angry at the computer ___ angry at yourself ___
helpless/resigned
___ determined to
fix it ___neutral ___
other: ___________
10.
How
many minutes did it take you to fix this specific problem? (if this has happened before, please account
only for the current time spent) _____________________________
11.
Other
than the amount of time it took you to fix the problem, how many minutes did
you lose because of this problem? (if
this has happened before, please account only for the current time lost; e.g.
time spent waiting or replacing lost work). ____________
Please explain:
12. Until this problem was
solved, were you able to work on something else?
____Yes ____No
Please explain:
Post-Session Survey (Appendix C)
Email address:
__________________________________
For the following questions, please circle the
number that best corresponds to your feelings.
1. Right now, my mood is:
Very Unhappy 1 2 3 4 5 6
7 8 9 Very Happy
2. We asked you to record your frustrating
experiences. Overall, how frustrated are you after these experiences?
Not Frustrated at
All 1
2 3 4 5 6
7 8 9 Very Frustrated
3. How will the frustrations that you experienced
affect the rest of your day?
Not at All 1
2 3 4 5 6
7 8 9 Very Much
4. Are the incidents that occurred while you were
recording your experiences typical of your everyday computer experience?
Yes No
5. In general, do you experience more or less
frustrating incidents while using a computer on an average day?
Less 1
2 3 4 5 6
7 8 9 More
6. Did these frustrating experiences impact your
ability to get your work done?
No impact 1
2 3 4 5
6 7 8 9 Severe impact
7. Did
these frustrating experiences impact your interaction with your co-workers?
No impact 1
2 3 4 5 6
7 8 9 Severe impact