Dec. 1995

Abstract

Multimedia is an art world term, often credited to designers Charles and Ray Eames, that describes the fusion of media such as painting, sculpture, photography, music, and video. Within the world of computers it is used broadly to describe almost any combination of media, ranging from simple text and graphics through to the Eames' vision (Nielsen, 1995; Preece et al., 1994; Shneiderman, 1992).

The diversity of multimedia user interfaces raises questions about the origin of these varied species. We think Darwin would have nodded his head knowingly. His theory of evolution by natural selection through survival of the fittest elegantly explains the huge diversity of organisms that occupy the numerous ecological niches. But can this natural world theory explain the evolution of widgets, interface styles and emerging genres of multimedia? Why have some survived while others have not? Where do multimedia design ideas come from and what determines their evolution?

Mutation, the raw material of evolution, does not exist outside the natural world but the pull of new technology and changing user demands stimulates novel designs. Then market forces determine survival, as companies and products come and go. Darwin would probably have been astounded by the rapid expansion of multimedia within only a few generations of computers. Predicting change is hard but we now know much about the strengths and limitations, likes and dislikes of humans interacting with computers. Principles can be drawn upon, which help to explain the survival of some interface features and extinction of others. For example, interfaces that are predictable and consistent, allow users to easily undo their actions, protect against errors and provide useful help at the right time tend to survive.

We identify seven multimedia eras: Nascent, Control, Construction, Ubiquitous, Collaborative, 3-D Virtual, and Visualization, and make tentative predictions for the future. As in nature, these era co-exist and over-lap, and there are also sudden 'ice-age' transitions.

Around 1980, at the dawn of the personal computer age, the primordial soup of multimedia consisted of green screens and videodisc images on separate monitors. This Nascent Era produced many chaotic and short-lived species with rigid interfaces that left the users frustrated victims of machines that they could not control. Examples include the five-minute video without a stop button or choice sequences which could not be reversed or cancelled. These species died out quickly because of their poor usability. As advancements in high resolution displays and fast chips spread, still and then moving images, animations, and sound flourished. The evolutionary force coming from both technical development and the demands of users, particularly video game and home computing enthusiasts followed by advertising, films and education assured their future.

In parallel with these developments interface complexity grew and users needed better and more direct ways of controlling them. This gave rise to the Control Era in which direct manipulation became the dominant interface form. Instead of modal dialogs and rigid sequencing users could make choices selecting objects as they saw fit, reversing, cancelling, reviewing, and confirming their actions as they wished. 'What you see is what you get', more commonly referred to as WYSIWYG, became a guiding principle with the world of action displayed visually and keyboards giving way to pointing, selecting, dragging, and stretching. The aim was to make operations rapid, incremental, and reversible and to prevent user errors by effective designs. For example, when selecting a date on a calendar it is impossible to make a syntactical input error. As direct manipulation interfaces became prevalent and so did machines with better graphics variants prevailed in which objects were designed with visual affordances (Norman, 1989) suggesting how to use them. Buttons, for example, looked as though they should be pushed.

In later generations of the Control Era during the late 80s and early 90s, embedded menus in text and graphics kept user attention on the contents and provided smooth hypertext linking (Koved & Shneiderman, 1986). The mouse became the device of choice, but trackballs, joysticks, and tablets with pens found successful niches. High precision touch screens with lift-off activation enabled the emergence of effective public access kiosks and creative finger gesturing.

The Construction Era also developed in which an increasing number of people became involved in some form of multimedia authoring (Shneiderman & Kearsley, 1989). Numerous authoring languages emerged in almost every ecological niche but were slow to spread and regularly died out, because they were cumbersome to use. Robust species support integration of text and images and construction tools for individual media (music, photos, drawings, video capture, etc.). On the other hand, simple and powerful tools to cut and paste video with dynamic text overlays, create and alter animations, synchronize music with images, or search multimedia are still rare and beautiful to behold.

In the mid 90s we are witnessing the Ubiquitous Era dawning with the growing availability of World Wide Web access with embedded menus providing links across the world. These developments have generated a frenzy of writing home pages and torrent of browsing. The remarkable potency of access to the net has led to an unusually rapid growth of web servers, applications, and usage. Isolated computer users may soon find it difficult to survive. In the early generations of this era the emphasis has been on surfing the net (reading, browsing, navigating), but in later generations greater facility in authoring web pages and delivering applications across the web will emerge. The awkwardness of separate viewers for video or external applications for animations will fade as integrated layouts become dominant in future web browsers.

Predicting evolutionary developments is a risky venture, but a new Collaborative Era seems likely. Email, once the delight of computer junkies, is now so prolific that overwhelmed gateways produce faltering businesses, raging children, cursing academics, and wailing lovers. Only the telecommunications companies and their shareholders smile. The prevalence of UseNet groups stimulate an increase in electronic text-based communication, but the seduction of video conferencing continues. Crude video conferences will become smooth, and participants will conveniently integrate simultaneous viewing and constructing of multimedia objects. Users will switch from seeing each other to conferencing over photos, maps, videos, documents, soundtracks, and animations with multiple cursors all in motion. They will discuss changes and any participant will make edits that are viewed by all.

Another likely prediction is the 3-D Virtual Era. While early tools show cluttered displays, obscured data, slow updates, and inadequate navigation, novel way finding techniques, better layouts, and faster processors will help. Full immersion, although good for a limited number of specialist tasks like fighter pilot training, is unlikely to spread too far because the cumbersome helmets, sweaty gloves and web of wires are intrusive. Desktop virtual reality, where users replace ëbeing-iní with ëlooking-atí seem more likely to flourish.

A final prediction is that the Information Visualization Era will facilitate presentation and exploration of multidimensional, relational, hierarchical, tabular, and temporal data. Information abundant and perceptually rich displays such as treemaps, starfields, table lenses, magic lenses, hyperbolic trees, fisheye views, timelines, will be explored with dynamic queries widgets, even across the net.

Across the eras, the responsiveness of systems (response times, display rates, transmission times, etc.) has increased, even while the image resolution and sound quality has improved, and the size of accessible multimedia databases has soared. While these trends seem likely to continue, the key principles for survival are mainly those of good usability and fulfilling a real need. However, market forces can be cruel and fickle. Resistance to novelty can slow down even robust worthy innovations with wealthy backing. But the defense provided by intellectual property protection is only sometimes a reliable shield against invaders. Stolen ideas, while frustrating to originators, can promote their evolution and survival!

Another change is that the user community has not only expanded but also diversified with a wider range of users and the distinction between users and developers becoming increasingly blurred. Programmers have been joined by graphic designers, filmmakers, historians, teachers, musicians, artists, designers and poets in creative teams. Content experts have gained exciting opportunities to tell their stories.

Evolution doesn't have a destination or a plan, each innovative gene has a chance to prove itself. For those of us who design innovations, the thrill is there every day as we create ever more viable user interfaces. Ultimately the forces and whims of the market place will drive evolution with history judging our success.

References

Koved, L. and Shneiderman, B., Embedded menus: Selecting items in context, Communications of the ACM 29, 4 (April 1986), 312-318.

Nielsen, J., Multimedia and Hypertext: The Internet and Beyond, Academic Press, Cambridge, MA, (1995).

Norman, D. A., The Psychology of Everyday Things, Basic Books, New York, (1988).

Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., and Carey, T., Human-Computer Interaction, Addison-Wesley, Reading, MA, (1994).

Shneiderman, B., Designing the User Interface: Strategies for Effective Human-Computer Interaction: Second Edition, Addison-Wesley Publishing Co., Reading, MA, (1992), 573 pages.

Shneiderman, B., and Kearsley, G., Hypertext Hands-On! An Introduction to a New Way of Organizing and Accessing Information, Addison-Wesley Publishing Co., Reading, MA, (1989).

Web Accessibility