October 23, 1996

Surf & Turf

Seeing the Forest of Knowledge for the Trees of Data

There is an important lesson about omniscience in the rolling countryside of New England.

From the top of many hills in the springtime, you can look out across the landscape and see a soft undulating sea of green flowing from your feet. It is almost a fluorescent green, punctuated by only a few sharp, white steeples that point, in that particularly human way, along the Y-axis, toward heaven.

From above the trees, the steeples tower over not only the living but also the dead resting below in the town cemetery.

These steeples, along with pyramids and obelisks, are a few of the ways that human beings have tried to represent the concept of omniscience -- a slippery idea of many contradictory parts that only humans would believe they could capture with wood and glass.

But there are two fundamental perceptions about omniscience that seem to flow through these representations -- perspective and persistence. We may not be anywhere close to the actual truth of omniscience, but these two concepts are as a good a place as any to start. What they suggest is that knowing all requires a presence in both space and time.

These two facets of omniscience are, in many ways, the fundamental issues of the Information Age. They are the keys to unlocking what Vannevar Bush called the record of knowledge. He believed that only through the mastery of this record could human beings move into the next evolutionary stage in intellectual development.

It has taken millennia for humans to master the previous eras of knowledge accumulation and distribution. The next era hinges on the ability to associate the billions of loose threads of data into a useable form.

The next era depends on the power of the computer. But what has become clear is that our ability to manage the flood of information lags far behind. Our concepts of how to manage this flow are still drawn from desktop paradigm that was outlined 15 years ago by Xerox researchers.

User interfaces have evolved enormously over the years, starting with the first teletypes and monitors, which broke through the physical limitations of paper through the use of vertical scrolling. The next great advance came with Xerox's Star interface, which added depth to the desktop. And in the past few years, virtual reality spaces have opened the door to complete three-dimensional navigation.

The next steps are not nearly so obvious. Within just the past few years, a series of research projects have been launched to look for new types of interfaces that strike at this concept of virtual omniscience. They are, in general, clunky, ugly and Unix-only. But they embrace powerful ideas that expand our perceptions about human beings and our relationship to the sum of our knowledge.

One of the most interesting now under way tackles an aspect of interface perspective that for lack of a better term, I call the focal point.

The name of the interface is PAD++ and it is being developed by James Hollan and Ben Bederson at the University of New Mexico and Ken Perlin and David Fox, among others, at New York University under the partial sponsorship of the Defense Department's Advanced Research Projects Agency.

Every computer interface, from the old command lines to virtual reality spaces, has used a single focal point that lies in the general area of where the user's head is. Every aspect of these interfaces is adjusted to match that point, from font size to window placement.

What PAD++ does is to allow the user to simultaneously adopt many points of perspective with an infinite variety of focal lengths. Although it uses a traditional desktop metaphor, it is really based on the idea of the video camera.

The building block of the interface is something called a portal, which looks like a standard window but functions like a video camera that can be pointed at various spots on the desktop. The desktop itself is vast. Instead of limiting its space to the size of the monitor, it extends far beyond the screen in all three dimensions

For example, let's say that we have constructed a virtual desktop about the size of Yankee Stadium. We can see only one screen of the desktop, but we can place portals on the screen to monitor the entire stadium, first base and right field in miniature form.

Navigating through the entire area can be accomplished by clicking on spots in the portals or moving the mouse in the desired direction while holding a button down.

Lenses in the Pad++ interface allow users to look at information in different ways -- a set of numbers viewed through the pink lens, in this example, produces a bar graph. Through the blue lens, the user sees a scatter plot.

The key mechanism of the portal is a feature called data zooming, which functions in a similar way to zooming with a video camera. Click on an area of a portal and it begins to expand as if you were moving closer to it. One of the key aspects of zooming is that the object moves toward you smoothly, just as it would if you zoomed in with a video camera. The smooth zooming helps to maintain the sense of direction and the context of the information on which you are focusing.

The designers of PAD++ have gone one step further with the concept of zooming. Not only can you zoom in on visual objects, but also on data. For example, you could zoom in on a personnel file and a picture of the employee would pop up. Zoom in on the picture, and you would see a resume. Zoom in on a single resume entry and description of the company would expand in the portal.

The cinematic paradigm is continued through the inclusion of what are called lenses. The lens concept, developed at Xerox, functions in much the same way as camera lenses and filters. In PAD++ they are essentially portals that transform the user's view of data. For example, passing a lens over a document in French could translate it into English. Passing a lens over a plain map could illuminate various types of information: one lens could be used to show roads, another the weather and yet another, the topography of the area.

One of the main weakness of PAD++ is that it is essentially only a set of tools. It has no grand metaphor of how to organize information. In other words, it gives you the means to navigate around a large virtual desktop, but not an ideology to organize the information in the way that, say, Windows 95 or the Macintosh OS do with folders.

There hasn't been much improvement on the topical system of folders in 15 years. It is a system that makes sense when the amount of information to be categorized is relatively small. But in world of hypertext, where files cut across many boundaries, it begins to fall apart into an abysmal maze of folders inside folders inside more folders.

A group of researchers at Yale has begun looking at a solution that continues the dimensional expansion of the interface by moving to the last frontier - into the fourth dimension, time.

The name of the project is Lifestreams. It is, in essence, a diary of everything you do in the virtual realm. All the information that passes through your computer is automatically placed into chronological streams of overlapping pages.

There are two key concepts in Lifestreams: data persistence -- that is, nothing is ever destroyed or deleted -- and data neutrality, the idea that nothing falls into a category until you order it that way.

The point is to maintain a system that views information in the broadest and most neutral way while maintaining the context in which it was first created or received.

When you start your computer, a main stream of pages is displayed on your screen that recedes into the past. The top of the stack is the present, although you can also have the stack move forward into the future, which would contain things like to-do lists and reminders.

The stream includes every chunk of data you have ever touched. Each chunk is called an event, and all the events added together form the fundamental structure of your virtual life. Rough navigation of the main stream is accomplished through a scroll bar that moves you through time, although the designers of the system have used other metaphors, including a calendar.

You can work on any document by clicking on it. If you change the document or create a new one, that event is bumped to the top of the main stream. The original remains in its location in the mainstream. Filtering the information into useable categories is done through the find box. Type in a key word and a sub-stream is created that includes all the related documents.

The beauty of Lifestreams is its simplicity in both operation and concept. Instead of categorizing information in folders and sub-folders, there is only the stream. The current interface has the time scroll bar, a few buttons for copying or creating documents, and a find box. But it can function almost the same with nothing more than the find box and the stream of pages. Simplicity, in this case, is beauty.

Its greatest power is not the chronological ordering of information, which can be just as cumbersome as folders when you are dealing with a few thousand events. What is significant is the idea that everything has to be saved. The chronology is only a rough framework to hold this concept and to avoid one of the great headaches of the Information Age -- trying to figure out into which folder to stick things.

The eternality of information is the building block of the association. It is an idea that breaks down the fixation on the present and turns time into a seamless whole.

For all its conceptual beauty, there are weaknesses to Lifestreams that stem from its neutral and featureless view of time. Every page looks the same. Every stream flows in the same way. Every year looks the same as another.

This seems at odds with human perceptions of time, in which some moments are more significant than others, some years more action packed and memorable. It also produces a natural clumping of information that may not fit into folders very well but doesn't just lie there hanging around either. What I think is lacking is a way to show the full landscape of rivers and rivulets, and the natural clumping of some events into ponds and lakes.

Eric Freeman, the researcher who designed Lifestreams, said that that he considered adopting some aspects of PAD++, like lenses and data zooming, so that the total stream could be seen from different perspectives. He has also been thinking of ways to highlight sub-streams so the grouping of some events is more apparent.

Both PAD++ and Lifestreams are in the early stages of development and are certain to evolve. But what is clear from their ideas is that traditional methods of perceiving information will be dead in future. In a world of pedabyte and exabyte knowledge, folders are no closer than steeples to representing omniscience.

While steeples have great meaning and importance, in many ways, it has always been the forest surrounding them that is more instructive on this issue. Within the web of foliage are connections that are far beyond our ability to grasp. An yet, despite its complexity, there is a wholeness and order to the confusion that can be felt down to the smallest detail.

The forest is a wonderful reminder that it may not be our lot ever to achieve omniscience -- in this world or the virtual one. It is simply not our destiny. But like the steeples that poke through the canopy of trees, it is often just the pointing alone that is significant.

SURF & TURF is published weekly, on Wednesdays. Click here for a list of links to other columns in the series.

Related Sites
Following are links to the external Web sites mentioned in this article. These sites are not part of The New York Times on the Web, and The Times has no control over their content or availability. When you have finished visiting any of these sites, you will be able to return to this page by clicking on your Web browser's "Back" button or icon until this page reappears.

Ashley Dunn at welcomes your comments and suggestions.

Home | Sections | Contents | Search | Forums | Help

Copyright 1996 The New York Times Company