Performance Benefits of Simultaneous over Sequential
Menus
As Task Complexity Increases
H. Hochheiser, B. Shneiderman*
Human-Computer Interaction Lab, Dept. of Computer
Science,
*Institute for Systems Research &
Institute for Advanced
Computer Studies
University of Maryland, College Park, MD 20742 USA
Abstract
To date,
experimental comparisons of menu layouts have concentrated on variants of
hierarchical structures of sequentially presented menus. Simultaneous menus - layouts that present
multiple active menus on a screen at the same time – are an alternative
arrangement that may be useful in many web design situations. This paper
describes an experiment involving a between-subject comparison of simultaneous
menu and their traditional sequential counterparts. Twenty experienced web
users used either simultaneous or sequential menus in a standard web browser to
answer questions based on US Census data. Our results suggest that appropriate
use of simultaneous menus can lead to improved task performance speeds without
harming subjective satisfaction measures. For novice users performing simple
tasks the simplicity of sequential menus appears to be helpful, but experienced
users performing complex tasks may benefit from simultaneous menus. Design improvements can amplify the benefits
of simultaneous menu layouts.
Keywords
Menu Design, Sequential
Menus, Simultaneous Menus, User Interfaces, Human-Computer Interaction,
Empirical Studies, Experimental Comparison
1. Introduction
Despite the
proliferation of drop-down menus and other enhancements, most modern web sites
use one of two strategies to support navigation and location of desired
information resources. Hierarchical or sequential menus present choices that
must be made in some predetermined order, with the impact of a given choice
constrained by the sum total of all previous choices. Query-based, form
fill-in interfaces use input widgets to support search based on a specified set
of attributes. This paper discusses a
third possibility – simultaneous menus –
which may be appropriate for design situations that are not well suited for
sequential or query based menu systems.
Sequential menus
(Figure 1) are most appropriate for situations requiring context-dependent menu
choices, such as choosing a continent, then a country, then a city, to get a
list of tourist attractions. However,
the rigidity of hierarchical menus causes difficulties for some tasks,
particularly when explorations and comparisons among the results of multiple
selections are required. To complete
such tasks with a hierarchical menu layout, users must make repeated choices
involving repeated backtracking through the hierarchy. Sequential menus may
also lead to disorientation: without appropriate contextual information, users
may find themselves lost in the menu structure.
Figure
1: Sequential Menus. Users must make
once choice from each menu in succession. Users can select the “Return to …” links
to revisit the previously displayed menu, or the “Reset Menus” link to return
to the first menu selection.
Query-based,
form-fill-in interfaces are frequently provided for searches of
non-hierarchical, multi-attribute data sets. Familiar examples include airline
reservation sites, “power searches“ on search engines, and on-line automobile
sales. These forms are particularly useful when users must select options from
a large range of alternatives, as drop-down menus, and list boxes can be used
to support query specification in a compact space. For searches based on
user-provided text, form-based interfaces can use text input boxes for query
terms, providing additional power with the risk of increased user errors.
Despite these
advantages, query-based interfaces often suffer from some of the serialization
problems associated with sequential menus. Searching is often provided in a
“batch“ mode – a search is submitted, results are displayed, and the user must
return to the search screen to make another search. As a result, comparison
between results of searches may be cumbersome. Forms must also provide
appropriate contextual cues: form output displays that fail to indicate the
values of the search parameters may disorient users.
Simultaneous menus
(Figure 2) are an alternative design possibility appropriate for tasks that do
not involve context-dependent modification of menu contents or unconstrained
text input. These menus, which simultaneously display choices from multiple
levels in the hierarchy, provide users with the ability to make choices from
the menu in any order, for example choosing continents, primary language, and
types of tourist attractions to get a list of cities with their attractions.
Simultaneous menus are
similar to query-based interfaces, in that both offer a variety of choices that
can be made independently. However, simultaneous menus have the advantage of
supporting simple and straightforward comparison between options, as a single
selection in any of the menus is sufficient to move from one data point to the
next. This flexibility may lead to improved performance or user satisfaction
for some tasks.
Simultaneous menus have
some drawbacks. Effective use of this strategy depends upon the availability of
screen real estate necessary for display of the appropriate menu choices, so
simultaneous menus may not be appropriate for very broad (or very deep) menu
structures. Furthermore, simultaneous menu structures that display large
amounts of information the available screen space may require additional mouse
movement and/or cognitive processing that could offset improvements in
performance.
While systems such as
the National Digital Library collection browser [15] and Spotfire [19] used
simultaneous menus, evaluation has been limited. One study found that tasks
involving simultaneous menus took less time and had fewer errors than tasks
involving hierarchical menus [17]. The authors of this study hypothesized that
the use of a stable spatial presentation of the menus eliminated the need for
repeated visual scanning, thus reducing the cognitive load of a larger display.
This paper presents an
experiment that compares user task performance times for sequential vs.
simultaneous menus. Our hypotheses were that simultaneous menus would have faster
performance times and greater user satisfaction than sequential menus.
Furthermore, the performance advantage of simultaneous menus should increase
with the number of menu choices required to complete a task.
Figure
2: Simultaneous menus. Users can choose from any one of the three menus on the
left at any point in time.
2. Related Research
Numerous experimental studies have been conducted to clarify tradeoffs
affecting performance in menu usage. Much of this work has focused on the
breadth vs. depth tradeoff in design of sequential menus systems for navigating
through hierarchies. Empirical studies conducted by Miller [11], Snowberry,
Parksinson, & Sisson [18], Kiger [7], and others [6,8,20] compared user
performance with menu structures of various depths and breadths. These studies have consistently found that
increases in menu depth lead to increased task completion times and error
rates. Jacko and Salvendy [6] examined
the relationship between task complexity and performance for menus of various
breadths and depths. Building on Campbell’s model of task complexity [1], they
found that response time and number of errors increased as menu depth
increased. Furthermore, users found deeper menus to be more complex.
The consistent performance advantages for broad, shallow menus indicate
that designers should favor control structures with fewer steps. Deep, narrow menus structures distribute
selections over a greater number of menus, resulting in increased task
performance times, perceived complexity [6], risk of disorientation, and cost
of error recovery. Broad, shallow menus
avoid these difficulties by reducing the number of steps necessary even though
there are more choices at each step. One possible implication of the superior
performance of broader menus is that additional performance gains might be
realized by presenting all choices at once. This is the motivation behind
simultaneous menus.
Recent studies, focusing on web-based systems, have confirmed the
benefits of broader, shallower menu structures. Zaphiris and Mtei [21] examined user performance in a system
based on WWW hyperlinks. Using hierarchies similar to those used by Kiger [7],
they found that the two-level structures were faster than three or four level
structures.
Larson and Czerwinski [9] raised several concerns regarding earlier work
and its applicability to web-based systems. Specifically, many existing web
sites have link fan-out significantly greater than the ranges used in earlier
breadth vs. depth studies. Furthermore,
earlier studies often used the same categories across all structures, leading
to potentially unnatural menu categories. These concerns were addressed by a
study using broader structures and menu contents that were designed to be
natural. This study involved the comparison between layouts consisting of eight
choices at each of three levels (83), with 16 choices followed by 32
(16x32), and 32 choices followed by 16 (32x16), with menu structures based on
an editor’s design of category contents that both fit the desired structure and
appeared natural. They found that the two-level hierarchies performed better
than the three-level (83) version, with no significant differences
in user preference between the three layouts.
Larson and Czerwinski also questioned the effect of training on menu
performance, and applicability to web sites. After observing that earlier
studies provided users with the ability to study hierarchies and learn from
mistakes, the authors note that the continual evolution of web sites may deny
users the opportunity to realize performance gains that may be associated with
training [9]. Although their study does not address this issue directly, they
raise an important issue: transferability of earlier studies on menu designs to
web sites is likely to be dependent upon a clear understanding of the
similarities and differences between the environments involved.
Several investigators
have moved beyond questions of breadth vs. depth to investigate the impact of
other features of menu structure design and layout. Parkinson, Sisson, and
Snowberry [14] examined the impact of layout decisions such as spacing or no
spacing between category groups, alphabetical vs. categorical ordering of
options within category groups, and arrangement by column or row. They found
that spacing between categories and columnar organization independently and
significantly improved performance. Alphabetical vs. categorical ordering
within categorized groups did not affect performance.
Norman and Chin [13]
examined the effect of menu structure shape. Using a 256-item menu structure
and a constant depth of four, they compared structures with a range of items at
each level, including constant (4x4x4x4), decreasing (8x8x2x2), increasing (2x2x8x8),
concave (8x2x2x8) and convex (2x8x8x2). For “scenario targets” involving a
search for items that met a set of specified criteria, concave menus were shown
to have the best performance.
Another perspective on the impact of menu structure and presentation can
be found in Zaphiris, Shneiderman, and Norman’s [22] study of the use of
expandable menus in web environments. Expandable menus – layouts which use
in-place expansion to present hierarchical choices in context – were expected
to improve user performance and reduce backtracking. However, for hierarchies
of depth two, three, and four, sequential menus had faster response times.
Expandable menus did not reduce backtracking. In fact, trials with expandable
menus and hierarchy depth of four had more backtracking than sequential menus,
although this result was not significant.
Empirical research has provided evidence supporting the use of
simultaneous menus. In a study involving monitoring of functional variables of
simulated machines, Seppälä and Salvendy [17] compared “parallel” presentation
of menus to three different hierarchical presentations. Tasks were categorized
in four distances, which varied the number of changes that users would need to
make in order to move between targets. In all cases, the parallel menus had
faster task performance and lower rates.
Simultaneous menus are
closely related to query preview interfaces [3]. These environments combine widgets for specifying query
constraints with feedback describing the size of the result set meeting those
constraints. Simultaneous menus may be seen as a special case of query
previews, in which all data is immediately available without execution of a
database query.
Further insight into menu structure and design issues can be found in
analytic models and simulations of performance with various layouts. Lee and
MacGregor [10] combined analytical models based on the number of alternatives
per page, key press times, assumed reading rates, and computer response times
with simulations. They concluded that the optimal number of alternatives per
menu page is usually less than 10, and may be as low as 4 to 8. It is not clear
how these models might be reconciled with the empirical results indicating
improved performance with greater menu breadth.
In other work, Hornof and Kieras used simulation models to predict
performance for both ordered and unordered pull-down menus. For unordered
menus, they found that models involving parallel processing of multiple menu
items using both random and systematic strategies provided the most accurate
predictions [4]. For ordered menus, models accounting for use of motor
preparation based on approximate known location information best account for
observed data [5]. Although these studies are based on the use of pull-down
menus and therefore may not be directly applicable to web environments, they
provide an illustration of the complex interaction of multiple cognitive and
motor issues that may be involved in menu operation.
3. A Task-Based
Predictive Model
Intuitively, the simultaneous
menu layout would appear to have the advantage of freeing users from making
selections in a pre-determined order. For simple tasks involving a single
selection from each menu, this may lead to a performance improvement. However, the real benefits of simultaneous
menus are likely to be seen in tasks that require revisitation of menus in
order to compare results of different choices.
To see why this is so,
we imagine a set of three menus, and three types of questions, involving
varying levels of difficulty. Type 1 tasks require one selection from each of
the three menus. Type 2 tasks require
comparison between two selections, which differ only in the choice made from
the third menu. To complete this task,
the user must traverse the menu tree once, note the appropriate result, and
make a second selection from the last menu to select the appropriate comparison
data. Finally, type 3 tasks are similar
to Type 2 tasks, but the selections differ only in the choice made from the
second menu.
For simultaneous menus,
these more complicated tasks involve minimal additional overhead: as all menus
are constantly available, the user can simply move to the appropriate menu and
make the desired choice. However, these tasks place sequential menus at a significant
disadvantage, as users must explicitly “backtrack” to return to a previously
displayed menu and make a new choice. Thus, our expectation was that simultaneous
menus would lead to faster task performance than sequential menus and that
performance advantages of simultaneous menus would be greater for tasks
involving more backtracking through the menu structure.
A simple “clicks model”
[2], based on the number of clicks required for the task types described above,
will provide a more specific understanding of the predicted performance
differences:
• Type 1: For both menu layouts, users must make one
selection in each of the three menus, for a total of three clicks.
• Type 2: Users must make one selection at each of the
three levels, plus appropriate clicks to get the second data point. For
simultaneous menus, this involves one additional click on the third menu, for a
total of four clicks. For sequential menus, one click of the “Back” button is
required, along with one additional click on the third menu, for a total of
five clicks.
• Type 3: For
simultaneous menus, four selections are necessary, as these questions only
alter one of the three menu choices. However, sequential menus require seven
clicks: five as required for type two, plus one Back click and a new menu
selection at the second level.
These results are
summarized in Table 1. The “Items Varied” for any given task is the
total number of menu choices that changes. For type three, two backtracking
steps are required in the sequential case even though only one item is varied:
these questions require a different choice on the second menu, while requiring
the same choice for both visits to the third menu. Thus, these questions are somewhat easier for simultaneous menu
users, who need only make one additional selection from the second menu to
complete the task.
|
Task |
# Items Varied |
# Backtrack Steps |
Simultaneous Clicks |
Sequential Clicks |
|
Type 1 |
0 |
0 |
3 |
3 |
|
Type 2 |
1 |
1 |
4 |
5 (=3+1+1) |
|
Type 3 |
1 |
2 |
4 |
7 (=3+2+2) |
Table 1: Summary of the three task types. Type
1 included no backtracking, so three clicks were needed for both sequential and
simultaneous menus. Type 2 questions involved
two choices from the third menu, thus requiring two additional clicks for
sequential menu users (one to return to the previous menu, and one to make a
second choice), and one additional click for simultaneous menus. Finally, Type
3 questions varied the second category. For simultaneous menus, this added only
the one click required to make the additional choice, so these questions are no
harder than those in Type 2. However, sequential users had to make four
additional clicks: two to return to the second menu, one to make a new choice
from that menu, and one to repeat the selection made from the third menu.
Further analysis can generalize the contents of Table
1 into a predictive model based on the number of clicks required to complete
each task. For simultaneous menus, users must make one selection at each of the
initial menus, followed by an additional click for each comparison that must be
made. If we refer to the result of one
complete traversal of the menu sequence as a single data point, the total
number of comparisons that must be made is one less than the number of data
points that must be accessed: to make one comparison, I must access two data
points, etc. Thus, tasks involving the use of 
simultaneous menus to compare data from 
data points will require a total of 
clicks.
For sequential menus, the number of backtracking steps
is the crucial factor in determining the number of clicks required to complete
a task. To see why this is so, we first note that one choice from each menu
will be necessary to view the first data screen. After those choices are made, each backtracking step involves two
additional clicks: one to return to the previous menu, and a second click to
make a selection from the menu to which the user was backtracking. Thus, 
backtracking steps require 
additional clicks, for a total of 
clicks (
is the number of menus, as above).
For both menu types, we assume that each menu
selection action takes a given time - 
for simultaneous
menus or 
for sequential. Each task involves a constant (possibly
zero) initiation time - 
or 
. Combining these observations, we derive the following
equations:
Simultaneous: 
,
Sequential: 
As above, 
is the number of menus (three in our examples), 
is the number of
screens that must be compared in the simultaneous case, 
is the number of backtracking steps required in the
sequential case, and 
and 
are constants determined by the type of menu layout being
used (sequential or simultaneous). Expanding terms, these equations become:
Simultaneous: 
Sequential: 
For any given set of menus, the second and third terms
of these equations will be constants.
The present experiment uses the number of backtracking
steps in the sequential case as a measure of complexity. This measure implies
that Type 3 questions are more complex than Type 2 questions, even though they
require the same number of clicks for users of simultaneous menus. This measure
accounts for one of the primary advantages of simultaneous menus: as all menus
are displayed concurrently, a single change in any menu is not dependent upon
the ordering of the menu. This is in stark contrast with the difference cost of
changing menu selections in the sequential model, as we see in question Types 2
and 3. In those cases, a single change
in the second menu (type 3) is more expensive than a change in Type 2. However,
in the simultaneous case, we do not expect Type 3 to be significantly more
expensive than Type 2 questions.
These models assume that all clicks take approximately
equal amounts of times. A more complete model would include predictors for
times required to make choices from each of the 
menus. Such a model would build upon research showing that
menu selection times can be roughly logarithmic or linear [8,12,16]. Although
we expect that selection times for individual menus used in this experiment
will conform to these earlier findings, item selection times for individual
menus may differ when used in different layouts. Specifically, the increased amount of information on the
simultaneous menu screen may lead to greater cognitive load, causing item
selection times to be greater than for sequential menus. However, the increased number of
re-orientations required may slow users of sequential menus.
4. Experiment
Informal
investigation and the above predictive model led us to hypothesize that users
of simultaneous menus would be able to complete tasks in less time than users
of comparable sequential menu layouts. Furthermore, simultaneous menus should
show increasing performance advantages as task complexity increases. Although
other dependent factors – specifically learning time and accuracy – might be
measured, they were not addressed in the current experiment.
Our experiment used
the three question types described above to provide three separate types of
tasks. Each participant answered
questions using one of the two menu layouts.
The experimental task consisted of 15 questions, divided evenly among
the three types described above. Task completion times were aggregated by menu
type and task type, and mean times for the three types were compared by menu
layout.
Experimental tasks were
based on data taken from the U.S. Census Bureau’s MapStats web page (http://www.census.gov/datamap/www/index.html). County business patterns profile data for
1993-1996 provided a data set covering 23 counties, nine industries, and four
years. These attributes formed the basis for a three-menu layout, with the
sequential menu layout displaying counties first, industries second, and years
third.
Each combination of
county, industry, and year had three corresponding facts: annual payroll, number of employees, and
number of establishments. This formed the basis for the questions, which
required retrieving individual facts (“How many people were employed in Kent
County in service businesses during 1993?”) or comparing between two different
data points (“Which business category employed the larger number of people in
Howard County in 1995: manufacturing or wholesale trade?”).
The experiment involved
a total of 21 questions, split evenly among the three question types described
in Section 3. Thus, one-third of the
questions required one selection from each of the three menus, one-third required
an additional selection from the third (year) menu, and the remainder required
an additional selection from the second (industries) menu. The differences
between these tasks are comparable to the four distances used in an earlier
evaluation of simultaneous menus by Seppälä and Salvendy [17]. Of the 21 questions, six were practice
questions and experimental data was taken from the 15 remaining questions.
Practice and experimental questions were presented in a balanced order
consisting of sets of three questions, with each set containing one question of
each type.
The menus were presented
to users as HTML hyperlinks displayed in a Netscape browser. In the simultaneous menu case (Figure 2),
menus were displayed in three frames on the left-hand side of the browser
window, while a frame on the right-hand side contained the results, or text
asking the user to make a choice from any menus that have not yet been
selected. After selections were made, the relevant menu windows would refresh
to highlight the selected item. At any time, the user had the option of
selecting the “Return to Start” link, which would reset the menus to their
original configuration.
Sequential menus (Figure
1) were presented in a series of three screens, each containing one of the
three menu items. Users moved forward
in the menu sequence by simply selecting a single item from a menu. Two types
of links supported returning to previously viewed menus in the sequence: a “Return to Start” link cleared the
selection state of the system and returned to the initial menu screen, while a
“Return to …” link on the second and third menu screens was available for
moving back to the previously displayed menu. All menu screens (with the
exception of the first menu) contained feedback mechanisms summarizing the
choices that had been made on previous menus.
In order to eliminate
variation due to network delays, files were served from a web server running
locally on the machines used for testing. Browser cache functionality was
disabled, in order to guarantee that each menu request generated a page request
to the server. As this configuration guarantees an entry in the server log file
for each menu selection, request timestamps in the server logs were used to
extract task performance times.
Twenty-two volunteer subjects
participated in the experiment.. Data
collected for two of the users of sequential menus was not used in the
analysis, as tasks completion times for these users were several times greater
than for other users. These subjects
were both older than the other participants and less experienced with web
browsers. These differences in background presented the possibility of other
factors that may have influenced task performance. Furthermore, inclusion of
this data may have artificially skewed the statistical analysis towards
favoring simultaneous menus. To avoid these artifacts, we eliminated this data
from the analysis.
Of the remaining 20
participants, 15 were male, 5 were female, and all were under 45 years of age.
All subjects were graduate, undergraduate students, or technical staff, and all
had previous web-browsing experience. Subjects were randomly assigned to use
either sequential or simultaneous menus: of the twenty data sets used in the
analysis, eleven involved simultaneous menus while the other nine used
sequential menus
Participants began their
experimental sessions by signing the consent form, completing the background
questionnaire, and reading a one-page instruction document appropriate for the
menu layout being used. After indicating their understanding of the
instructions, users completed the practice tasks, took a short break if needed,
and continued on to the experimental tasks. Finally, users completed a short
post-experimental questionnaire aimed at understanding their subjective reactions.
This questionnaire consisted of eight questions, asking users to rate the
screen layout, system navigation, information arrangement, amount of
information displayed, and initial instructions on a 1-9 scale.
Task presentation and
completion were handled identically for both menu layouts and phases of the
experimental session (practice and experimental questions). All questions were presented to users on a
sheet of paper, which was also used to record the answers. Each task began with
the browser screen on a page containing a single link labeled “Next
Question”. Users were instructed to
read the question first, and to select the link only after they had completely
read the question. Selection of this link led to display of the appropriate
menu screen, allowing the users to navigate the menus to find the appropriate
data. Users were instructed to continue until they found the information needed
to answer the question, at which point they should write the answer on the
sheet. After writing the answer, users chose the link marked “Next Question”,
which returned the browser to the initial start page, ready for the next
question. The elapsed time between the
selections of the “Next Question” links was recorded as the time required to
complete the task. The instructions, task presentation, and menu layout were
all revised to account for feedback from a pilot test with four subjects.
This experiment measures
task performance times for users who are unfamiliar with simultaneous menus. In
order to understand the potential performance as users become more comfortable
with simultaneous menus, three of the authors completed the experimental tasks
three times for each menu layout. The best time for each question was extracted
from the resulting data set, and the results were averaged across question
type, creating an estimated performance profile for proficient users. Although
less formal and thorough than an experimental evaluation of learning effects,
this analysis provides some insights into the possible benefits of simultaneous
menus for experienced users.
5. Results
Table 2 and Figures 3
and 4 summarize the results.
|
Question Type |
Menu Layout |
Min |
Max |
Average (n=20) |
Std. Dev. |
Proficient User (n=3) Estimates |
|
Type 1 |
Sequential |
5 |
36 |
14.6 |
5.9 |
8.4 |
|
|
Simultaneous |
12 |
43 |
21.3 |
6.1 |
7.8 |
|
Type 2 |
Sequential |
9 |
41 |
25.1 |
8.1 |
15.0 |
|
|
Simultaneous |
12 |
72 |
29.3 |
8.7 |
10.2 |
|
Type 3 |
Sequential |
12 |
69 |
39.4 |
12.7 |
22.2 |
|
|
Simultaneous |
15 |
62 |
33.5 |
9.6 |
13.2 |
Table
2: Summary of Task Completion Times, in seconds. For the experimental results,
sequential menus were faster for types one and two, and simultaneous menus were
faster for type three. In the estimated proficient user profile, simultaneous
menus were always faster. In all cases,
task completion times increased as complexity increased (complexity is defined
as the number of backtracking steps for the simultaneous case).
Average task times for
each task type/user combination were used for statistical analysis. As
expected, task type had a significant impact upon performance (F(2,36)=67.17,
P<0.001). The menu presentation
style was not a significant factor (F(1,18)=.68,P>0.05), but the interaction
between menu style and task type was (F(2,36)=8.65,P=0.01). Thus, although
sequential menus appear to have benefits for simple tasks, simultaneous menus
are preferable for complex tasks, and the advantages of simultaneous menus
increase with task complexity. This result supports the use of simultaneous
menus for tasks involving comparisons between the results of multiple menu
selections.
Figure 3:
Experimental Results: Sequential menus produced better performance for Types 1
and 2, but simultaneous menus were faster for Type 3 (n=20). Error bars
indicate a range of one standard deviation from the mean.
Figure
4: Estimated performance results for proficient users, based on composite
results from three experienced users: Simultaneous menus are faster for all
three types of questions, indicating a possible learning effect that may favor
simultaneous menus.
This data includes times
for incorrect responses: we assume that the subjects took the time and did the
appropriate page navigation even if the final answer was incorrect.
Results from the estimated performance profile based on the experienced
users are given in Figure 4.
Simultaneous menus outperformed sequential menus for all three task
types, and performance differences increased with task complexity. Although the informal nature of this data clearly
limits the conclusions that can be drawn, these results provide preliminary
evidence for increased advantages of simultaneous menus for more experienced
users.
Subjective
responses to the two menu types were similar. Individual t-tests for each of
the eight questions showed no significant differences (at the 0.05 level)
between the responses for the two different menu types. Users found both
simultaneous and sequential menu layouts to be somewhat satisfying, easy to
use, and easy to navigate. Additional evaluation - particularly involving
individuals with less computer experience – may clarify user preferences, but
the lack of a clear trend of confusion or disorientation among users of the
simultaneous menu layout is encouraging. A within subjects design would be more
likely to show preference differences.
Figure
5: Subjective Questionnaire Results: Average values are shown for 20 subjects,
with error bars indicating one standard deviation difference. Higher numbers
are better results. Roughly comparable subjective responses to the two menu
types provide some indication that users are not necessarily confused or
disoriented by simultaneous menus.
6. Predictive Model
Revisited
Our models assume a
single item selection time for all menus used in a given layout: for any single
layout (simultaneous or sequential), selection times for the three menus
(counties, industries, and years) should be comparable. For both sequential and
simultaneous windows, analysis of data for the individual menus showed similar
item selection times. In both cases, selection times for the county and year
menus were significantly shorter than times for the industry menus, and no
significant differences between the county and year menus were observed. This
result is somewhat surprising, since the industry menu had fewer items (9) than
the county menu (23).
Two possible
factors might explain this performance difference. The ordering of the choices
within each menu may have been a factor. While county names where presented in
alphabetical order, industry names were presented in the essentially random
order used on the census web site. A more likely explanation can be found in
the cognitive load involved in processing the menu choices may have been a
factor: while county names are short (one or two words) and possibly familiar,
industry names involved greater amounts of text with which participants were
less likely to be familiar, such as “Agricultural Services, Forestry, and
Fishing”, or “Transportation and Public Utilities”.
The performance
selection times for the second and third menus may explain the surprising
differences between performance times for Type 2 and Type 3 for simultaneous
menus. Since tasks of both of these types involved only one additional
selection in either the second (Type 3) or third (Type 2) menu, we expected the
task performance times to be comparable. The observed slower performance on
Type 3 tasks is consistent with the observation that menu choices from the
second menu took significantly longer than choices from the third menu.
In any case, the
absence of any evidence of a relationship between menu lengths and item
selection times provides initial justification for the assumption of a single
item selection time for each layout.
To examine the fit
between our data and the predictive models given above, we conducted a
regression of the task completion times against the number of data points
compared (for simultaneous menus), or the number of backtracking steps required
(for sequential menus). For the simultaneous
menus, the linear regression equation was 
, 
. For sequential menus, the linear equation was 
, 
. A quadratic regression fits the data equally well -
, 
- suggesting the
possibility of a non-linear effect for sequential menus. A more thorough
characterization of this effect would require more data, but it seems possible
that the time required for a given backtracking step might be influenced by the
number of preceding backtracking steps, thus leading to a non-linear effect.
On average, the choice time required for a
menu in a simultaneous layout is greater than the time required for the same
menu in a sequential layout. We can use the average menu selection times for
each of the menu layouts to relate the linear equations back to the predictive
models presented above. For simultaneous menus, the average selection time was
4.9 seconds (
), and the average for sequential menus was 3.0 seconds (
).
Using these values,
and the depth of the menu structure used (
), we can identify the constants that match the calculated
regressions. Specifically, for the simultaneous menus, we find 
and 
, so 
Using the average
value 
this becomes 
, where 
is the number of data
points visited. Similarly, for sequential menus, we find that 
and 
so 
, or 
, where 
is the number of backtracking steps required to complete a
task.
Although these results
support the use of a clicks-only predictive model, further work will be needed
to validate these models. Specifically, the interaction between menu type and
task type observed in the statistical analysis suggests that the advantage for
simultaneous menus may grow as tasks become more difficult. We believe that this is the result of the
increased cognitive load of repeated backtracking while comparing multiple data
points, which is likely to be difficult for sequential menu users.
Additional experiments
involving a wider range of backtracking steps and required mouse clicks might
clarify the time functions for both menu types. More accurate accounting for the time to read menus and make
choices could lead to a deeper understanding of the components of task
completion times. Inclusion of
appropriate models of mouse motion and distance (perhaps based on Fitts’ Law)
could account for the effects of screen layout. Finally, investigations of learning rates could lead to models
that predict improvements in task performance.
7. Discussion
These results suggest that task complexity – as measured by the amount
of backtracking required for sequential menus - is likely to be the largest
factor in performance differences between simultaneous menus and sequential
menus. For simple tasks that do not require comparisons between multiple result
sets, sequential menus are faster.
However, the advantage shifts for tasks requiring more than one
backtracking step (for the sequential menus), and the observed interaction
between task type and menu type suggests that the advantages of simultaneous
menus may become even greater for tasks involving more comparisons.
The change in the relative advantages of these menu types may be
explained by the relationship between cognitive load and task type. For simple
tasks, sequential menus provide simpler screen layouts that avoid the clutter
and possible confusion of simultaneous menus. However, simultaneous layouts may
reduce time and effort spent reading menus. As the simultaneous menus are
constantly displayed on the screen, users may learn the menu contents faster
and thus reduce the cost of re-scanning menus to make additional selections.
For sequential menu layouts, a menu is removed from the screen after a choice
is made and remains inaccessible until the user returns from that menu. Users
may need to re-read menu contents with each visitation, thus slowing learning
and decreasing performance.
In terms of Campbell’s task complexity framework [1], the multiple paths
that might be used to reach the goal may cause the simultaneous menus to be
more complex, and therefore slower, for simpler (type 1) tasks. For types 2 and
3, the increased complexity associated with multiple outcomes is greater, and
may overshadow the complexity due to multiple paths. Furthermore, Jacko and
Salvendy’s results linking greater depth with increased complexity [6] provide
reason to believe that the complexity cost of additional outcomes is likely to
be greater for sequential menus, thus leading to the performance advantage for
simultaneous menus seen for Type 3 tasks. Of course, additional investigation
and analysis would be needed to validate these models.
Simultaneous menus are generally unfamiliar, even to experienced users.
Our experienced user performance profile suggests the possibility of an
advantage for simultaneous menus across all task types. Although further
experimentation will be needed to understand learning effects, training and
practice might help users take advantage of simultaneous menus for both simple
and complex tasks.
Minor design decisions
can have significant impact on performance with menu layouts [6,14]. Artifacts of our experimental design may
have influenced our results, and suggest interesting modifications for further
study:
Screen Layout:
For the sequential menus, each menu appeared in the screen in the space
occupied by the previous menu. This layout minimized the mouse movement
required: all tasks could be completed in the upper left quadrant of the
screen. On the other hand, the simultaneous menus were displayed in a vertical
column of three frames, occupying the entire left half of the screen. The extra
movement required may have degraded performance. Compact menu arrangements
could reduce the extra mouse movement required for navigation of simultaneous
menus.
Familiarity of Menu Presentation Style: Since most of the participants were undergraduate
and graduate students with substantial experience using web browsers, it seems
reasonable to conclude that the sequential menu format was well understood. As
simultaneous menus are unfamiliar, there may be a learning effect involved in
the results.
Task Choice:
All three task types in this experiment involved closed-end questions with
known answers. However, the performance advantage of simultaneous menus
relative to sequential menus increased with the amount of backtracking
required. Repetition of this experiment with tasks that involve more
backtracking may lead to results that are still more favorable for simultaneous
menus.
Menu item ordering: The second menu (containing industry names) contained items that were
significantly longer than the items in the other two menus. Furthermore, the
items in this menu were presented in an arbitrary (non-alphabetized) order. These disparities may have contributed to
the performance differences between task Types 2 and 3. In particular, these
artifacts may be responsible for the unexpected differences between Types 2 and
3 for users of simultaneous menus.
Simultaneous menus fared well on the post-test subjective questionnaire.
Ratings for the two menu arrangements were roughly comparable on all of the
subjective questions. This provides preliminary support for the conclusion that
simultaneous menus do not appear to confuse or disorient users. Since each
subject used only one of the two menu presentation styles, a true preference
comparison between the two styles is not possible. Further study involving
within-subjects comparison of the two menu styles might clarify issues related
to user preference while providing additional data for performance comparisons.
Ideally, any further work along these lines would involve tasks that are well
motivated and more realistic than the tasks used for this study. The resulting
data would provide a more robust picture of the impact of simultaneous menus on
task performance time and user satisfaction.
The comparison between simultaneous and sequential menus is necessarily
limited: as the most familiar, “traditional” style of menu item presentation,
sequential menus provided a baseline for evaluation of simultaneous menus.
Additional experimentation aimed at comparing simultaneous menus with
query-based forms and other menu presentation styles might provide a clearer
model of the tasks that might benefit from the use of simultaneous menus.
8. Design
Implications
When sequential menu hierarchies can be converted to simultaneous menu
presentation, this strategy should be considered if complex or exploratory
tasks are anticipated. Simultaneous menus
show users all alternatives at all levels at once, thereby aiding comprehension
of all possibilities, although the increased perceptual and cognitive load may
slow novice users in simple tasks.
Simultaneous menus usually require more display space, which may render
them inappropriate for certain display environments and menu structures. This increased screen content may lead to
further increases in perceptual load, which could have additional negative
effects. Furthermore, simultaneous
menus present a tradeoff between menu space and data space. Menu configurations
that cannot be displayed simultaneously in a manner that leaves sufficient
screen space for results may be more appropriate for sequential presentation.
Compact presentation formats might be used to present simultaneous menus in a
manner that minimizes these detrimental effects. Interface widgets such as sliders or checkboxes may provide more
effective and compact representations of menu choices.
For tasks requiring comparisons among multiple data points, simultaneous
menus might be most helpful when used in conjunction with other techniques for
supporting navigation. For example, dynamic content generation might be used to
display multiple results in an appropriate display. The combination of simultaneous display of multiple results sets
with simultaneous menus might be used to provide easy access to multiple data
points with mechanisms that support comparisons between these results.
The user population may
influence the choice between simultaneous and sequential menus. Our experimental data, which was collected
from subjects unfamiliar with simultaneous menus, showed an advantage for
sequential menus on simpler tasks. However, this advantage disappeared in our
estimated profile based on more experienced users, suggesting that the benefits
of simultaneous menus are likely to increase with user experience.
9. Conclusion
We have shown that
simultaneous menus can lead to improvements in user performance over comparable
sequential layouts. The choice between simultaneous and sequential menu layouts
should be made on the basis of the expected task: if users are expected to make
multiple selections from two or more menus, simultaneous menus provide better
performance. Simultaneous menus appear
to be well suited for exploratory tasks, since they also provide a continuous
overview of menus at all levels.
Much of the menu design
literature has focused on analysis of the breadth vs. depth question in
hierarchical menu structures. Although clearly important, these investigations
present an overly simplistic view of the problem of menu structure design. Our
study presents one alternative to strictly hierarchical menus, along with
evidence that simultaneous menus can lead to improved performance.
Comparisons that limit
the parameters of menu designs to depth and breadth may not account for some
factors that affect performance. Studies of menu structures with different
shapes [13] and with differing amounts of contextual information [2, 22] have
shown that performance can be influenced by the type of task, the amount of
context given, and the shape of the menu.
Examination of these issues as they apply to simultaneous menu is a
promising direction for future work.
This paper does not
address the relative performance of form-fill-in menus. Additional
experimentation comparing simultaneous menus to form-fill-in menus might
clarify the utility of the various approaches. Hybrid layouts – perhaps
combining simultaneous menus with form fill-in or query preview [3]
functionality - present intriguing possibilities and additional areas for
future exploration.
Acknowledgments
Thanks to Kent Norman
for his invaluable help with experimental design and statistics. Richard Salter
provided assistance with HtX (http://www.cs.oberlin.edu/~rms) for generation of
the HTML pages needed to run the experiment. Natasha Kositsyna and Geoffroy
Ville provided invaluable assistance with data collection and analysis. Mary
Czerwinski, Andrew Sears, and the anonymous reviewers provided helpful comments
that were greatly appreciated. We appreciate the partial support of the US
Census Bureau and IBM’s University Partnership Program.
References
[1] Campbell, D. (1988)
Task complexity: a review and analysis. Academy
of Management Review, 13, 40-52.
[2] Chimera, R. and
Shneiderman, B. (1994) An exploratory evaluation of three interfaces for
browsing large hierarchical tables of contents. ACM Transactions on
Information Systems 12, 4 , 383–406.
[3] Doan, K., Plaisant, C., Shneiderman, B., and
Bruns, T. (1999) Interface and data architecture for query preview in networked
information systems. ACM Transactions on
Information Systems 17, 3, 320-341.
[4] Hornof, A. and
Kieras, D. (1997) Cognitive modeling reveals menu search is both random and
systematic. Proc. CHI ’97 Conference:
Human Factors in Computer Systems, ACM Press, NY, 107–114.
[5] Hornof, A., and
Kieras, D. (1997) Cognitive modeling demonstrates how people use anticipated
location knowledge of menu items. Proc.
CHI ’99 Conference: Human Factors in Computer Systems, ACM Press, NY,
410-417.
[6] Jacko, J. and
Salvendy, G. (1996) Hierarchical menu design: breadth, depth, and task
complexity. Perceptual and Motor Skills,
82, 1187-1201
[7] Kiger, J. (1984) The depth/breadth trade-off in the
design of menu-driven user interfaces.
International Journal of
Man-Machine Studies 20, 201–213.
[8] Landauer, T. and
Nachbar, D. (1985) Selection from alphabetic and numeric trees using a touch
screen: Breadth, depth, and width. Proc.
CHI ’85 Conference: Human Factors in Computer Systems, ACM Press,
NY, 73–78.
[9] Larson, K. and
Czerwinski, M. (1998) Web page design: Implications of memory, structure, and
scent for information retrieval. Proc.
CHI ’98 Conference: Human Factors in Computer Systems, ACM Press,
NY, 25–31.
[10] Lee, E. and
MacGregor, J. (1985) Minimizing user search time in menu retrieval
systems. Human Factors 27, 157–162.
[11] Miller, D. (1981)
The depth/breadth tradeoff in hierarchical computer menus. Proceedings of 25th Annual Meeting - Human Factors Society, NY,
296-300.
[12] Norman, K. (1991)The
Psychology of Menu Selection: Designing Cognitive Control of the Human/Computer
Interface. Ablex Publishing Corporation,
Norwood, NJ.
[13] Norman, K. and
Chin, J. (1988) The effect of tree
structure on search in a hierarchical menu selection system. Behaviour
and Information Technology 7, 1, 51–65.
[14] Parkinson, S.,
Sisson, N., and Snowberry, K. (1985) Organization of broad computer menu
displays International Journal of
Man-Machine Studies 23, 689-697.
[15] Plaisant, C.,
Marchionini, G., Bruns, T., Komlodi, A., and Campbell, L. (1997) Bringing treasures to the surface:
Iterative design for the library of congress national digital library program. Proc. CHI ’97 Conference: Human Factors in
Computer Systems, ACM Press, NY, 518–525.
[16] Sears, A. and
Shneiderman, B. (1994) Split menus: Effectively using selection frequency to
organize menus. ACM Transactions on Human-Computer Interaction 1, 1 , 27–51.
[17] Seppälä, P. and
Salvendy, G. (1985) Impact of depth of
menu hierarchy on performance effectiveness in a supervisory task: Computerized
flexible manufacturing system. Human Factors 27, 713–722.
[18] Snowberry, K.,
Parksinson, S., & Sisson, N. (1983). Computer display menus. Ergonomics
26(7), 699-712.
[19] Spotfire. (1999) Spotfire [WWW Document]. URL http://www.spotfire.com.
[20] Wallace, D.,
Anderson, N., & Shneiderman, B. (1987) Time stress effects on two menu
selection systems. Proceedings of 31st
Annual Meeting – Human Factors Society, NY, 727-731.
[21] Zaphiris, P. and
Mtei, L. (1997) Depth. vs. breadth in the arrangement of web links. URL http://otal.umd.edu/SHORE/bs04.
[22] Zaphiris, P.,
Shneiderman, B., and Norman, K. (1999) Expandable indexes versus sequential
menus for searching hierarchies on the world wide web. URL
http://www.kypros.org/ pzaphiri/Paper.