Between 1987 and 1991 the University of Maryland's Human-Computer Interaction Laboratory worked on a series of projects which shared a common aspect: the use of touchscreens. The first pole of research was the use of touchscreens for information kiosks. This was closely tied to our research on Hypertext and Hyperties (the Guide of Volunteered Archaeology video below shows Hyperties.) As information was becoming easier to navigate with hypertext links, information kiosks were becoming more important for museums and other public places, but mice were unpractical in public settings, so touchscreens were the natural choice!
During 1989-1991, we worked on a home automation system and explored several direct manipulation designs (e.g. clocks and calendars to schedule devices to go on and off, ON-OFF switches with buttons or sliders) . A playful fingerpainting exploration tool and toy called Playpen was developed. Finally we worked with NCR to explore how touchscreens could be used to replace keyboards when minimum and occasional data entry were needed. We stopped doing research on touchscreens as successful applications found their ways in museums or cash registers. Pen interfaces, made popular by the Palm Pilots, continued the work as they afford a similar sense of "real" direct manipulation. Eventually touchcreens came back as the input device of choice of mobile devices, especially after the launch of the iPhone is 2007.
From bad reputation to high precision touchscreen
For a quick summary of our work refer to:
For a longer review of the state of the art of touchscreen use at that time see: Shneiderman, B. (March 1991), Touch screens now offer compelling uses,
IEEE Software 8, 2, (March 1991) 93-94, 107.
Also in Sparks of Innovation in Human-Computer Interaction, Shneiderman, B., Ed., Ablex (June 1993) 187-193.
To produce the first HCIL Video Report in 1991 we recorded many of our demonstrations from 1988-1991 (several are now on YouTube and embedded below). Those videos have been used extensively in HCI classes. Sears, A., Plaisant, C., Shneiderman, B., A new era for high-precision touchscreens (1990 tech report),
Later published in Advances in Human-Computer Interaction, vol. 3, Hartson, R. & Hix, D. Eds., Ablex (1992) 1-33.
In 1987 (and still long afterwards) touchscreens had the bad reputation of being imprecise. Most user interface books would state that touchscreens selections were "of course limited to targets larger than the average finger". To use touchscreens for browsing information systems such as Hyperties, we had to be able to select small targets (e.g. the letters of the alphabets of the index table of content). At the time, all touchscreens selections were done in such a way that a target was selected as soon as the finger came over it, and the corresponding action was performed immediately (we called it "first touch" or "land-on" strategy). Errors were common, due to parallax or calibration problems, and users were frustrated when the wrong target was repeatedly selected by mistake.
A first breakthrough was to propose an alternative technique for selection: the lift-off strategy. As users touch the screen, feedback is provided as to what will be selected and the action takes place when the finger is lifted off the screen. In our implementation a cursor was drawn on the screen slightly above the finger. When the cursor was over a target, the target was highlighted. Users could then either lift-off their finger to select the highlighted target, or adjust their position by sliding their finger to a neighboring target. This was a major breakthrough: only the cursor position mattered for the selection, not the finger itself. Selecting a single character was now possible.
The touchscreen technology has greatly improved but overall the lif-off strategy is still useful (e.g. the iPhone uses a lift-off strategy.)88-04 Potter, R.L., Weldon, L.J., Shneiderman, B. (May 1988)High-Precision touchscreen: the next step was to try to stabilize the touchscreen so that the cursor would stay put when the finger didn't move. This was accomplished with a clever time-dependant averaging of the positions returned by the device. Now, individual pixels could be selected (in the 480x350 high resolution screen of the time). An experiment showed that there was significant difference in selection times and error rates between mouse and touchscreen for targets down to about 1mm2, when using a lift-off strategy with a stabilized touchcreen. Companies such as Elographics and Microtouch, with whom we had good relations, integrated stabilization techniques into the drivers of their touchscreens. From then on, high-precision was possible, and designers could do everything with the touchscreen that they could do with the mouse.
Improving the accuracy of touch screens: an experimental evaluation of three strategies,
Proc. of the Conference on Human Factors in Computing Systems, CHI `88 (Washington, DC) 27-32.
Also Sparks of Innovation in Human-Computer Interaction, Shneiderman, B., Ed., Ablex (June 1993) 161-169.89-17 - Sears, A., Shneiderman, B. (June 1989)
High precision touchscreens: design strategies and comparisons with a mouse,
International Journal of Man-Machine Studies, (1991) 34, 4, 593-613.
Also Sparks of Innovation in Human-Computer Interaction, Shneiderman, B., Ed., Ablex (June 1993) 171-185.
Online Public Access Catalog (OPAC) for the Library of Congress
The first large test of our strategies for the use of touchscreens was in the Spring 1988 when a public access kiosk was designed using Hyperties for the Caesarea (King Herod's Dream) Exhibit, a Smithsonian Museum exhibit on archaeology. Usability testing in the museum was very important in identifying practical ways to make the touchscreen even more usable.
90-09 Plaisant, C. (Nov. 1990)
Guide to opportunities in volunteer archaeology - case study of the use of a hypertext system in a museum exhibit,
Hypertext/Hypermedia Handbook, Berk E. & Devlin, J., Eds., McGraw-Hill (1991) 498-505.
Also Sparks of Innovation in Human-Computer Interaction, Shneiderman, B., Ed., Ablex (June 1993) 223-229.
89-08 Shneiderman, B., Brethauer, D., Plaisant, C., Potter, R. (May 1989), Evaluating three museum installations of a hypertext, Journal of the American Society for Information Science, 40(3) 172-182.
Also Sparks of Innovation in Human-Computer Interaction, Shneiderman, B., Ed., Ablex (June 1993) 231-250.
We also worked with the Library of Congress to develop an online catalog using touchscreens for the main reading room of the Library.
Home Automation : Schedulers, toggles, etc.
A chapter of Sparks of Innovation tells the story:
Marchionini, G., Ashley, M., Korzendorfer, L. ACCESS at the Library of Congress (1993) .
Also related to information kiosks:89-03 Sears, A., Kochavy, Y., Shneiderman, B. (1989)
Touchscreen field specification for public access database queries: let your fingers do the walking,
Proc. of the ACM Computer Science Conference `90 (Feb. 1990) 1-7.
In the summer of 1988 we started working with a small Maryland Company called American Voice and Robotics. Their focus was on providing home owners with easy control over all major electric systems in the home via a computer equipped with a touchscreen mounted in the walls or cabinetry (there was a voice command, but rarely used, except for demos). Our designs fot that application included many novel examples of the use of touchscreens (selecting zones on maps, turning calendars or hands of watches to set time, or sliding objects on timelines).
Toggles (buttons, sliders, rockers etc.),
90-10 Plaisant, C., Shneiderman, B., Battaglia, J. (1990)
Scheduling home-control devices: a case study of the transition from the research project to a product, Human Factors in Practice, Computer Systems Technical Group, Human Factors Society (Santa Monica, CA, Dec. 1990) 7-13. Also Sparks of Innovation in Human-Computer Interaction, Shneiderman, B., Ed., Ablex (June 1993) 205-215
89-18 Plaisant, C., Shneiderman, B. (revised Feb. 1991)
Scheduling home control devices: design issues and usability evaluation of four touchscreen interfaces,
International Journal of Man-Machine Studies (1992) 36, 375-393.
In 1990 we designed and compared a series of touchscreen toggle switches allowing devices to be switched ON or OFF. The designs included button type toggles and sliding toggles.
This video was distributed to the attendees of the 1991 Human-Computer Interaction Laboratory Open House on June 7, and later on published in the SIGGRAPH Video Review, Issue 77 corresponding to the CHI '92 Technical video program.(see the accompanying CHI'92 Proceeding short paper). The video and its short paper are now being cited as prior art in legal cases contesting the Apple patents related to the touchscreen slider that unlocks the iPhone.
Designed originally for home automation, those toggles were also later used in a NASA design toolkit. Now slider designs can be found everywhere in touchscreen-based smartphones (such as the first iPhone in 2007). As we mentionned in our paper and video a slider design is more secure so that the phone cannot be turned on by mistake. A click confirms the action. Any design that enforces a sliding gesture would achieve a similar goal.
90-08 Plaisant, C., Wallace, D. (Nov. 1990)
Touchscreen toggle switches: push or slide? Design issues and usability study,
This tool/toy was developed by Andrew Sears to illustrate the possibilities of touchscreens (and to be a real fun demo!). In particular it demonstrated how touchscreens could do more than selection of targets using location as input, and that speed and direction could also be used for interaction.
The 1991 HCIL Video Reports contains a demo of playpen.
Touchscreen keyboards, a series of evaluations
Even before mobile devices touchscreen keyboards were attractive when only occasional data entry was needed. We worked with NCR (National Cash Register) to develop and test touchscreen keyboards as they were thought to be potentially useful to enter customer information on future touchscreen cash registers. Several experiments refined our designs. We were able to show a performance of 20 wpm for experienced users (10 wpm for novices)on a 3 inch wide touchscreen keyboard (mounted on computer monitor recessed in a desk).
91-12 Plaisant,C., Sears, A. (Sept. 1991)
Touchscreen interfaces for alphanumeric data entry,
Proc. of the Human Factors Society - 36th Annual Meeting, vol. 1, (Atlanta, GA, Oct. 12-16, 1992) 293-297. Also Sparks of Innovation in Human-Computer Interaction, Shneiderman, B., Ed., Ablex (June 1993) 195-204. Also Human Factors Perspectiv es on Human-Computer Interaction, Selections from Proc. of Human Factors and Ergonomics Society Annual Meetings 1983-1994, Perlman, G., Green, G.K., Wogalter, M.S., Eds. (1995) 261-265.
91-07 Sears, A., Revis, D., Swatski, J., Crittenden, R., Shneiderman, B. (April 1991)
Investigating touchscreen typing: The effect of keyboard size on typing speed,
Behavior & Information Technology, vol. 12, 1 (Jan-Feb 1993) 17-22. Drum copy
90-06 Sears, A. (revised March 1991)
Improving touchscreen keyboards: design issues and a comparison with other devices,
Interacting with Computers, vol. 3, 3 (1991) 253-269. First mention of our touchscreen keyboard:
89-03 Sears, A., Kochavy, Y., Shneiderman, B. (1989)
Touchscreen field specification for public access database queries: let your fingers do the walking
Proc. of the ACM Computer Science Conference `90 (Feb. 1990) 1-7.