High-Precision Touchscreens
1988-1991 HCIL Research

Explorations for Museum Kiosks, Device Control,
Finger Painting and Touchscreen Keyboards

Edited by Catherine Plaisant (last updated in Dec. 2011 - links fixed July 2013)

** News **
Our early work becomes hot again when cited in legal challenges to Apple's Slide to Unlock patent.
See the 2016 Apple-Sansung Court Decision in which "Plaisant" is cited 31 times as prior art for Slide to Unlock.
or see the fun 2011 video of the Story on UK CHANNEL 4 NEWS when it started to heat up.

Between 1987 and 1991 the University of Maryland's Human-Computer Interaction Laboratory worked on a series of projects which shared a common aspect: the use of touchscreens. The first pole of research was the use of touchscreens for information kiosks.  This was closely tied to our research on Hypertext and Hyperties (the Guide of Volunteered Archaeology video below shows Hyperties.) As information was becoming easier to navigate with hypertext links, information kiosks were becoming more important for museums and other public places, but mice were unpractical in public settings, so touchscreens were the natural choice.

During 1989-1991, we worked on a home automation system and explored several direct manipulation designs (e.g. clocks and calendars to schedule devices to go on and off, ON-OFF switches with buttons or sliders) . A playful fingerpainting exploration tool and toy called Playpen was developed. Finally we worked with National Cash Register (NCR) to explore how touchscreens might be used to replace keyboards when store cashiers needed to enter a little bit of data about shopppers (such as phone numbers or addresses). We stopped doing research on touchscreens as successful applications found their ways in museums or cash registers. Pen interfaces, made popular by the Palm Pilots, continued the work as they afforded a similar sense of "real" direct manipulation. Eventually touchcreens came back as the input device of choice of mobile devices, especially after the launch of the iPhone is 2007.

For a quick summary of our work refer to:

Shneiderman, B. (March 1991), Touch screens now offer compelling uses,
IEEE Software 8, 2, (March 1991) 93-94, 107.
Also in Sparks of Innovation in Human-Computer Interaction, Shneiderman, B., Ed., Ablex (June 1993) 187-193.
For a longer review of the state of the art of touchscreen use at that time see: Sears, A., Plaisant, C., Shneiderman, B., A new era for high-precision touchscreens (1990 tech report),
CS-TR-2487, CAR-TR-506
Later published in Advances in Human-Computer Interaction, vol. 3, Hartson, R. & Hix, D. Eds., Ablex (1992) 1-33.
We produced the first HCIL Video Report in 1991 by recording the demonstrations from 1988-1991 (most are now on YouTube and embedded below). Those videos were given along with copies of our papers to all the sponsors of the lab, and attendees of the annual HCIL symposium. The videos were available for sale, and have been used extensively in HCI classes. Many were also published as part of the ACM CHI videos.
Between 1988 and 1993 Apple was a sponsor of the HCIL lab, Steve Jobs visited in person in 1988, and Ben Shneiderman was a consultant for Apple at several occasions. Demos were also shown to our lab's visitors, and videos shown during invited lectures at conferences or during industry visits (see Ben Shneiderman's resume for a partial list)
  From bad reputation to high precision touchscreen

In 1987 (and still long afterwards) touchscreens had the bad reputation of being imprecise. Most user interface books would state that touchscreens selections were "of course limited to targets larger than the average finger".  To use touchscreens for browsing information systems such as Hyperties, we had to be able to select small targets (e.g. the letters of the alphabets of the index table of content). At the time, all touchscreens selections were done in such a way that a target was selected as soon as the finger came over it, and the corresponding action was performed immediately (we called it "first touch" or "land-on" strategy). Errors were common, due to parallax or calibration problems, and users were frustrated when the wrong target was repeatedly selected by mistake.

Lift-Off strategy
A first breakthrough was to propose an alternative technique for selection: the lift-off strategy. As users touch the screen, feedback is provided as to what will be selected and the action takes place when the finger is lifted off the screen. In our implementation a cursor was drawn on the screen slightly above the finger. When the cursor was over a target, the target was highlighted. Users could then either lift-off their finger to select the highlighted target, or adjust their position by sliding their finger to a neighboring target. This was a major breakthrough: only the cursor position mattered for the selection, not the finger itself. Selecting a single character was now possible.
The touchscreen technology has greatly improved but overall the lif-off strategy is still useful (e.g. the iPhone uses a lift-off strategy.)

88-04  Potter, R.L., Weldon, L.J., Shneiderman, B. (May 1988)
Improving the accuracy of touch screens: an experimental evaluation of three strategies,
Proc. of the Conference on Human Factors in Computing Systems, CHI `88 (Washington, DC) 27-32.
Also Sparks of Innovation in Human-Computer Interaction, Shneiderman, B., Ed., Ablex (June 1993) 161-169.
High-Precision touchscreen: the next step was to try to stabilize the touchscreen so that the cursor would stay put when the finger didn't move. This was accomplished with a clever time-dependant averaging of the positions returned by the device. Now, individual pixels could be selected (in the 480x350 high resolution screen of the time). An experiment showed that there was significant difference in selection times and error rates between mouse and touchscreen for targets down to about 1mm2, when using a lift-off strategy with a stabilized touchcreen. Companies such as Elographics and Microtouch, with whom we had good relations, integrated stabilization techniques into the drivers of their touchscreens. From then on, high-precision was possible, and designers could do everything with the touchscreen that they could do with the mouse.
89-17 - Sears, A., Shneiderman, B. (June 1989)
High precision touchscreens: design strategies and comparisons with a mouse,
International Journal of Man-Machine Studies, (1991) 34, 4, 593-613.
Also Sparks of Innovation in Human-Computer Interaction, Shneiderman, B., Ed., Ablex (June 1993) 171-185.
The first Information Kiosks:
Guide of Volunteered Archaeology (GOVA), for the Smithsonian Museums

The first large test of our strategies for the use of touchscreens was in the Spring 1988 when a public access kiosk was designed using Hyperties for the Caesarea (King Herod's Dream) Exhibit, a Smithsonian Museum exhibit on archaeology.  Usability testing in the museum was very important in identifying practical ways to make the touchscreen even more usable. This may very well have been the first touscreen museum kiosk, used by tens of thousands of patrons over several months, as the exhibit travelled to several locations over the US and Canada (for which a French version was created).

chap5.1_fig1_scaled chap5.1_fig2_scaled chap5.1_fig3_scaled
(GOVA demo starts at 2:37)

90-09  Plaisant, C. (Nov. 1990)
Guide to opportunities in volunteer archaeology - case study of the use of a hypertext system in a museum exhibit,
CS-TR-2559, CAR-TR-523
Hypertext/Hypermedia Handbook, Berk E. & Devlin, J., Eds., McGraw-Hill (1991) 498-505.
Also Sparks of Innovation in Human-Computer Interaction, Shneiderman, B., Ed., Ablex (June 1993) 223-229.
 
89-08 Shneiderman, B., Brethauer, D., Plaisant, C., Potter, R. (May 1989), Evaluating three museum installations of a hypertext, Journal of the American Society for Information Science, 40(3) 172-182.
Also Sparks of Innovation in Human-Computer Interaction, Shneiderman, B., Ed., Ablex (June 1993) 231-250.
Online Public Access Catalog (OPAC) for the Library of Congress

We also worked with the Library of Congress to develop an online catalog using touchscreens for the main reading room of the Library.

chap5.3_fig2_scaled

A chapter of Sparks of Innovation tells the story:
Marchionini, G., Ashley, M., Korzendorfer, L. ACCESS at the Library of Congress (1993) .

Also related to information kiosks:

89-03  Sears, A., Kochavy, Y., Shneiderman, B. (1989)
Touchscreen field specification for public access database queries: let your fingers do the walking,
Proc. of the ACM Computer Science Conference `90 (Feb. 1990) 1-7.
 
Home Automation : Schedulers, toggles, etc.

In the summer of 1988 we started working with a small Maryland Company called American Voice and Robotics. Their focus was on providing home owners with easy control over all major electric systems in the home via a computer equipped with a touchscreen mounted in the walls or cabinetry (there was a voice command, but rarely used, except for demos). Our designs fot that application included many novel examples of the use of touchscreens (selecting zones on maps, turning calendars or hands of watches to set time, or sliding objects on timelines).

Scheduling devices

chap4.5_fig1_scaled chap4.3_fig1_scaled chap4.5_fig3_scaled


  90-10  Plaisant, C., Shneiderman, B., Battaglia, J. (1990)
Scheduling home-control devices: a case study of the transition from the research project to a product, Human Factors in Practice, Computer Systems Technical Group, Human Factors Society (Santa Monica, CA, Dec. 1990) 7-13. Also Sparks of Innovation in Human-Computer Interaction, Shneiderman, B., Ed., Ablex (June 1993) 205-215

89-18  Plaisant, C., Shneiderman, B. (revised Feb. 1991)
Scheduling home control devices: design issues and usability evaluation of four touchscreen interfaces,
CS-TR-2352, CAR-TR-472
International Journal of Man-Machine Studies (1992) 36, 375-393.

Toggles (buttons, sliders, rockers etc.),
In 1990 we designed and compared a series of touchscreen toggle switches allowing devices to be switched ON or OFF. The designs included button type toggles and sliding toggles.

chap4_fig1_scaled


This video was distributed to the attendees of the 1991 Human-Computer Interaction Laboratory Open House on June 7, and later on published in the SIGGRAPH Video Review, Issue 77 corresponding to the CHI '92 Technical video program.(see the accompanying CHI'92 Proceeding short paper). The video and its short paper are now being cited as prior art in legal cases contesting the Apple patents related to the "Slide to Unlock" touchscreen slider that unlocks the iPhone.

Designed originally for home automation, those toggles were also later used in a NASA design toolkit. Now slider designs can be found everywhere in touchscreen-based smartphones (such as the first iPhone in 2007). As we mentionned in our paper and video a slider design is more secure so that the phone cannot be turned on by mistake. A click confirms the action. Any design that enforces a sliding gesture would achieve a similar goal.

90-08  Plaisant, C., Wallace, D. (Nov. 1990)
Touchscreen toggle switches: push or slide? Design issues and usability study,
CS-TR-2557, CAR-TR-521


PlayPen for painting

This tool/toy was developed by Andrew Sears to illustrate the possibilities of touchscreens (and to be a real fun demo!). In particular it demonstrated how touchscreens could do more than selection of targets using location as input, and that speed and direction could also be used for interaction.

chap4.3_fig4_scaled PlayPen portrait drawing sample PlayPen smiley scribble sample PlayPen writing sample

The 1991 HCIL Video Reports contains a demo of playpen.

Touchscreen keyboards, a series of evaluations

Even before mobile devices touchscreen keyboards were attractive when only occasional data entry was needed. We worked with NCR (National Cash Register) to develop and test touchscreen keyboards as they were thought to be potentially useful to enter customer information on future touchscreen cash registers. Several experiments refined our designs. We were able to show a performance of 20 wpm for experienced users (10 wpm for novices)on a 3 inch wide touchscreen keyboard (mounted on computer monitor recessed in a desk).

chap4.3_fig5_scaled

91-12  Plaisant,C., Sears, A. (Sept. 1991)
Touchscreen interfaces for alphanumeric data entry,
CS-TR-2764, CAR-TR-585
Proc. of the Human Factors Society - 36th Annual Meeting, vol. 1, (Atlanta, GA, Oct. 12-16, 1992) 293-297. Also Sparks of Innovation in Human-Computer Interaction, Shneiderman, B., Ed., Ablex (June 1993) 195-204. Also Human Factors Perspectiv es on Human-Computer Interaction, Selections from Proc. of Human Factors and Ergonomics Society Annual Meetings 1983-1994, Perlman, G., Green, G.K., Wogalter, M.S., Eds. (1995) 261-265.

91-07  Sears, A., Revis, D., Swatski, J., Crittenden, R., Shneiderman, B. (April 1991)
Investigating touchscreen typing: The effect of keyboard size on typing speed,
Behavior & Information Technology, vol. 12, 1 (Jan-Feb 1993) 17-22. Drum copy

90-06  Sears, A. (revised March 1991)
Improving touchscreen keyboards: design issues and a comparison with other devices,
Interacting with Computers, vol. 3, 3 (1991) 253-269.

90-01  Sears, A., Plaisant, C., Shneiderman, B. (June 1990 TR)
A new era for high-precision touchscreens
Advances in Human-Computer Interaction, vol. 3, Hartson, R. & Hix, D. Eds., Ablex (1992) 1-33.

89-17  Sears, A., Shneiderman, B. (June 1989 TR) High precision touchscreens: design strategies and comparisons with a mouse
International Journal of Man-Machine Studies, (1991) 34, 4, 593-613. Also Sparks of Innovation in Human-Computer Interaction, Shneiderman, B., Ed., Ablex (June 1993) 171-185.

The first mention of our touchscreen keyboards:
89-03  Sears, A., Kochavy, Y., Shneiderman, B. (1989)
Touchscreen field specification for public access database queries: let your fingers do the walking
Proc. of the ACM Computer Science Conference `90 (Feb. 1990) 1-7.
final version

 
 
 
 
 

Web Accessibility