CAR-TR-824 May 1996
CS-TR-3644
Abstract
What we see is not always what we get. This
is the problem when the underlying structure of an interface is
hidden from the user's view. Users high in Spatial Visualization
Ability (SVA), are quick to learn the contingencies of these relationships
and are not hindered by this problem. Low SVA users, however,
have difficulty visualizing these contingencies and often get
lost. We examined data for 97 undergraduate students to determine
whether revealing hidden contingencies though visual cues would
facilitate Low SVA users, enabling them to approach the level
of performance of High SVA users on a computerized path finding
task. It was found that increasing interface apparency does seem
to benefit all users, but particularly those with Low SVA.
Apparency of Contingencies in Single Panel Menus
We live in an era where access to information is vital and the ability to use the latest technology is essential to attaining that information. How can we best design the technology to accommodate the varying learning styles of each individual? First, it is important to recognize that there are, in fact, many different types of learning abilities. Bransford (1979) notes three: reasoning ability, verbal ability, and spatial ability. It is this last ability on which we will concentrate. Recent studies in the area of Spatial Visualization Ability (SVA) have commented on its importance as a factor in these individual differences (Hegarty & Sims, 1994; Juhel, 1991; Salthouse, Babcock, Mitchell, & Palmon, 1990). Studies have shown that SVA is heavily tied to an individual's ability to navigate through a hierarchical data base (Butler, 1990, Vincente, Hayes, & Williges, 1987) thereby indicating the importance of addressing the needs and concerns of those individuals who are not as adept with spatial visualization. As computers become necessary to complete even some of our most routine tasks, it is these users who will be at a real disadvantage. Second, we must try to understand what differentiates individuals within that ability. Lohman (1989) notes, with respect to SVA, that, "All subjects rarely solve figural tasks in the same way...Some subjects solve items on such (paper folding) tests by generating mental images that they then transform holistically" (p.346) These people are categorized as High SVA individuals who are especially proficient at "generating, retaining, and transforming mental representations..." (p. 346). He goes on to note that other individuals use less visual means to solve these problems. These individuals are often categorized as Low SVA individuals. Finally, we need to determine how to design the technology so as to address the specific needs of this population, since these are often the individuals who are at a disadvantage when it comes to navigating through the system.
This study will address individual differences in terms of spatial visualization ability. Vincente, Hayes, and Williges (1987) showed SVA to be a very strong and influential predictor of performance in searching an information database. Also, Norman & Butler (1989b) and Butler (1990) found strong correlations between SVA and ability to search a hierarchical database. What then, can be done to enable users with low SVA to increase their ease with using computers? One of the "problems" with current systems is entangled in one of the advantages. Computers make available great quantities of information, however, much of it is hidden and therefore difficult to access. What would make it easier to find that information? How can we design the system so as to lessen the variability among the users by removing potential roadblocks? Four techniques for making navigation through information easier have been suggested. They are: spatial metaphors, graphical user interfaces, interface apparency, and interface manipulability. Analogies and metaphors aid the user by providing a mental model of the system, and have been shown to be quite effective (Carroll & Mack, 1985). However, these techniques do not differentially benefit low SVA users, so that these users still lag behind users with high SVA (Butler, 1990). Graphical User Interfaces (GUIs) utilize the spatial metaphor to create physical representations of these metaphors. Again, graphics do not differentially assist those individuals with low SVA. It is even possible that they may increase differences due to the fact that they are visually based, forcing the user to navigate by visual skills only. Interface apparency and manipulation involve revealing hidden relationships -- making these relationships apparent, and thus "off-load the spatial processing of images from the user to the interface" (Norman, 1994, p.201) and then rearranging the image -- allowing the user to manipulate the image externally. Studies by Norman & Butler (1989a, also reported in Norman, 1991, p. 312-313) used graphical information to reveal hidden relationships with dramatic benefits. They looked at four conditions: (1) buttons only (no apparency), (2) buttons plus all links (non contingent apparency), (3) buttons plus all links to the goal (goal apparency) and (4) buttons with links from the start to the goal (start/goal apparency). Their findings showed that the first two conditions required the users to employ trial and error in order to reach the goal, since they were given no external information on hidden contingencies. However, subjects in the last two conditions reached optimal performance level quickly, thus "nullifying any differences in SVA" ( Norman, 1994, p. 201). This study attempts to partially replicate the Norman and Butler (1989a) study, concentrating on conditions 1 and 3 -- no apparency versus goal apparency. We hope these results will suggest methods for using interface apparency to reduce differences in SVA, specifically helping users with low spatial visualization ability.
For this experiment we will be looking at the dependent
measures of mean time to solution and mean number of moves.
We hypothesize that the difference between high and low SVA subjects
will be less in the Goal Apparency Condition than in the No Apparency
Condition. In addition, practice effects will be investigated
over trials. In the No Apparency condition, practice over trials
should result in the learning of underlying contingent relationships
and should depend on SVA. In the Goal Apparency Condition, performance
is predicted to jump quickly to an asymptotic level shared by
subjects of both high and low SVA.
Method
Participants
Ninety-seven undergraduate students enrolled in
introductory psychology courses at The University of Maryland
participated in this experiment. Subjects were randomly assigned
to either the Apparent condition (N = 49), or the Non Apparent
Condition (N = 48). The 20 female and 77 male participants
ranged in age from 17 to 28 years and took part in the experiment
in order to gain extra credit as part of their course requirements.
Design
A 2 x 2 design was used to investigate the interaction
and main effects of Apparency and Spatial Visualization Ability
(SVA). Apparency is a dichotomous variable described as either
Apparent or Non Apparent. In the Apparent condition the subjects
were given information, in the form of lines between buttons,
which would help them find the goal button (see Figure 1). In
the Non Apparent condition, the subjects were not given the lines
to the goal. For SVA, subjects took a 20 question pre-experimental
test of Spatial Visualization as measured by the VZ-2 cognitive
test (Ekstrom, French, & Harmon, 1976) as well as a post-experimental
test (for validation purposes) on the computer. A median split
for number correct from both the pre and post tests was used to
determine level of SVA. Subjects scoring at or above 27 (out
of 40) were assigned to the High SVA group and those below 27
were assigned to the low SVA group.
Materials
This experiment was run in the AT&T Teaching Theater, located at the University of Maryland in College Park (Shneiderman, Alavi, Norman, & Borkowski, 1994). The room contains 20 workstations all networked together and with two instructor's computers. Each workstation is an AT&T Globalyst Pentium-based computer with 16MB of RAM and 570MB hard disk, and all units are linked together using an AT&T Starlan network and through a Novell server, and in turn, linked to the Internet. A workstation is composed of a keyboard, a mouse, and a high resolution color monitor recessed into the desk to conserve space. Designed for noise reduction and comfort, the room has wall-to-wall carpeting and the computer units are stored in an adjacent room.
The software used for this experiment was Object
Plus that runs under Windows in the Teaching Theater.
Both the experimental program and the VZ-2 program were created
using Object Plus.
Procedure
Up to twenty subjects at a time participated in this experiment. Each was seated at a computer running the experimental program. Half of the computers were set up to run the Apparent condition of the experiment and the other half, the Non Apparent condition. At the beginning of the session, the experimenter gave some basic instructions pertinent to all subjects, however, once the experiment began, it was completely driven by the computer.
The subjects were first asked to fill in a brief
background information questionnaire (Age, Sex, High School G.P.A.,
SAT scores, Years of computer use). Next they had up to 6 minutes
to complete a 20 question multiple choice paper folding test of
Spatial Visualization Ability (the VZ-2).
Then each subject went through the 32 trials of the experiment
that consisted of four different screens, each with a different
goal --> Red, Yellow, Blue or Green. These four screens were
randomly displayed eight times each, for a total of 32 trials.
The object of the experiment was to select buttons, moving from
the leftmost column of buttons to the rightmost, which would bring
the subject to the goal button (see Figure 1). Not all of the
buttons would lead to the goal, only certain paths were defined
to reach it. If the subject chose a correct path, then a "CORRECT"
message was displayed and the subject was taken to the next screen.
Otherwise, the subject received an incorrect path message, and
by clicking on the reset button was given another chance to find
the goal. The subject could not move on to the next screen until
a correct path had been chosen. At the end of the 32 trials the
subject took a post-experimental VZ-2 test (for validity testing).
Finally, the subject was debriefed on the experiment and on the
particular condition.
Results
The times to solution from the 32 trials were averaged to determine the mean time to solution for the two conditions (1) Apparency (Apparent vs. Non Apparent) and (2) SVA Level (High versus Low), and their interaction. The 2 factor ANOVA revealed a significant difference for the main effect of Apparency (F(1,93)=33.405, p<.01), for the main effect of SVA Level (F(1,93)=10.385, p<.01) and for the interaction effect of Apparency and SVA Level (F(1,93)=4.876, p<.05). A comparison of means gave the following average number of moves: M(High, Apparent)=4.788,
M(High,Non
Apparent)=7.229, M(Low,Apparent)=5.481,
and M(Low,Non Apparent)=10.942,
as shown in Figure 2.
Next, the number of moves from the 32 trials were averaged to determine the mean number of moves for the two conditions and their interaction. The 2 factor ANOVA revealed a significant difference for the main effect of Apparency (F(1,93)=46.692, p<.01), and for the interaction effect of Apparency and SVA Level (High versus Low SVA) (F(1,93)=5.324, p<.05). The main effect of SVA Level was not significant for this test. A comparison of means gave the following average times: M(High,Apparent)=4.867, M(High,Non Apparent)=6.269, M(Low,Apparent)=4.723, and M(Low,Non Apparent)=7.556, as shown in
Figure 3.
Low, NonApparent Low, Apparent High, Non Apparent High, Apparent
Condition
Low,NonApparent Low, Apparent High, Non Apparent High, Apparent
Condition
Figure 3. Bar graph of the mean contrasts between Apparency and SVA Level for Mean Number of Moves with 95% confidence error bars
In comparing the two Apparency levels for practice
effects over trials, we found that the subjects in the No Goal
condition took longer to learn the task and, in fact, did not
reach the same level of speed as the subjects in the Goal condition,
as described by Figure 4, which shows trials 1-4 and the final
trial (32) for both the Goal and the No Goal conditions. The
first 4 points, which represent the mean number of paths in the
Goal condition, are close to the optimal performance as represented
by the last line (trial 32). For the No Goal condition, the first
4 points are much farther away from optimal performance and, in
fact, do not reach that goal, even in the last trial. It is also
important to note, the degree of variability in the No Goal condition
as compared to the Goal condition.
Trials: A1 A2 A3 A4......A32 N1 N2 N3 N4.....N32
In addition, we divided the subjects in the No Goal condition into High and Low SVA, and found that even though the High SVA subjects started out with more variability among them, and at a greater distance from optimal performance, they quickly (within about 3 trials) achieved close to optimal performance (that is, the level attained by the subjects in the Goal condition). However, the Low SVA subjects were more erratic -- getting worse and more variable on their third and fourth trials and never achieving optimal performance, even though they started off with better performance than the High SVA subjects (see Figure 5).
These last two sets of scores were calculated for the Mean number of paths, but not for Time to solution. The reason for this is that the two measures are highly correlated, as shown by the Pearson Product Moment Correlation Coefficient (r=.92), which is not surprising since the fewer moves the subject makes, generally, the less time it takes to make them.
An additional test that was performed which involved
the test-retest validity of the VZ-2 on-line test, given once,
prior to the experiment, and once after completion of the experiment.
This was found to be quite reliable with the Pearson Product
Moment Correlation Coefficient of r=.76.
Discussion
This experiment examined the use of revealing underlying
contingent relationships in order to assist users with navigation.
Revealing these contingencies is expected to lessen the time
it takes to reach the desired goals and to decrease the number
of moves needed to reach that goal (and avoiding erroneous paths).
In particular, though, this research is primarily directed at
users with low SVA. Whereas subjects with high SVA may find it
easier to "discover" hidden contingencies on their own
and may even enjoy the challenge of doing so, users with low SVA
would probably find this to be an inconvenience and might even
avoid using the technology for fear of getting lost.
Trials: N1 N2 N3 N4......N32 N1 N2 N3 N4.....N32
High SVA
Low SVA
Results of this experiment indicate that the subjects who were able to utilize the underlying contingent relationships revealed by the Goal Apparency condition performed significantly better than the subjects in the No Apparency condition. In fact, in the Goal Apparency condition, High and Low SVA subjects performed at approximately the same level (in fact, in terms of number of moves, the Low SVA subjects performed even better than the High SVA subjects, requiring fewer moves to reach the goal). In the No Apparency condition, there was a large discrepancy between Low and High SVA subjects with High SVA subjects constantly outperforming the Low SVA subjects. This finding would tend to indicate that the use of underlying contingent relationships does differentially benefit Low SVA users (however, it is fair to say that it also helps High SVA to a lesser degree).
In addition, practice effects measured over trials showed that the performance of the subjects in the Goal Apparency condition improved rapidly and was at a near optimal level by about the third trial for both High and Low SVA subjects. However, in the No Apparency condition, the learning of underlying contingent relationships depended solely on SVA level such that subjects with high SVA were able to eventually improve over trials to a near optimal level (but not as quickly as in the Goal Apparency condition), while the Low SVA subjects never attained that level of performance.
In general, these results indicate that revealing
hidden contingencies does provide a benefit to both groups of
users -- High and Low SVA. By giving information about the underlying
path structure both types of users are able to quickly follow
these paths to achieve their goal states. However, this is a
technique that will probably not be used as much by High SVA subjects,
since they are often able (and they often prefer) to recognize
those contingencies on their own. This research is targeted for,
and would benefit most, those subjects who do not feel comfortable
with navigation and who are not as able to visualize spatial relationships.
As it becomes more necessary to be able to traverse larger networks
of information, such as the World Wide Web (WWW), it will become
even more important to provide guideposts for these people so
as not to lose them along the way.
Acknowledgments
This work was supported by a grant from NSF (IRI
- Interactive Systems 9423184). We would also like to thank Project
Director, Walt Gilbert, and Tara Stachura for the use and scheduling
of the AT&T Teaching Theater, and our assistants Brad Bebee
and Randy Pagulayan for their help in the data collection process.
References
Bransford, J. D. (1979). Human Cognition: Learning, Understanding, and Remembering. Belmont, CA: Wadesworth.
Butler, S. A. (1990) . The effect of method of instruction and spatial visualization ability on the subsequent navigation of a hierarchical data base. (CAR-TR-488 and CS-TR-2398) Department of Psychology and the Human/Computer Interaction Laboratory, University of Maryland, College Park, MD
Carroll, J. M., & Mack, R. L. (1985). Metaphor, computing systems , and active learning. International Journal of Man-Machine Studies, 22, 39-57.
Ekstrom, R. B., French, J. W., & Harmon, H. H. (1976). Manual for kit of factor-referenced cognitive tests. Princeton, NJ: Educational Testing Service.
Hegarty, M. & Sims, V. K. (1994). Individual differences in mental animation during mechanical reasoning. Memory and Cognition, 22, 411-430.
Juhel, J. (1991). Spatial abilities and individual differences in visual information processing. Intelligence, 15, 117-137
Lohman, D. F. (1989). Human intelligence: An introduction to advances in theory and research. Review of Educational Research, 59 , 333-373.
Norman, K. L. (1994). Spatial Visualization -- A Gateway to Computer-Based Technology. Journal of Special Education Technology, 12, 195-205.
Norman, K. L . (1991). The psychology of menu selection: Designing cognitive control at the human/computer interface. Norwood, NJ: Ablex.
Norman, K. L., & Butler, S. (1989a). Apparency: Guiding sequential decision making by revealing inherent contingencies. Paper presented at the Society for Judgment and Decision Making, Atlanta, GA.
Norman, K. L., & Butler, S. (1989b). Search by uncertainty: Menu selection by target probability. (CAR-TR-432 and CS-TR-2230). University of Maryland, Center for Automation Research and the Department of Computer Science, College Park, MD.
Salthouse, T. A. , Babcock, R. L., Mitchell, D. R., & Palmon, R., et. al. (1990). Sources of individual differences in spatial visualization ability. Intelligence, 14, 187-230.
Shneiderman, B., Alavi, M., Norman, K., & Borkowski, E. Y. (1994). Windows of Opportunity in Electronic Classrooms. Communications of the ACM, 38, 19-24.
Vincente, K. J., Hayes, B. C., & Williges, R. C. (1987). Assaying and isolating individual differences in searching a hierarchical file system. Human Factors, 29, 349-359.