Occlusion in Two-Dimensional Displays: Visualization of Meta-Data
Jeremy Manson
jmanson@cs.umd.edu
University of Maryland
College Park, MD 20742
Abstract
Dynamic queries and starfield displays are important techniques for the visualization of large sets of data.
In starfield displays, unfortunately, it is frequently the case that some data points are located in the same two-dimensional position as others: this leads to the problem of hidden data, or occlusion.
The clear encoding of occlusion therefore becomes important in any such display.
This paper discusses several methodologies which can be used to
show occlusion in two dimensions.
It also discusses how these approaches might encode other information about the
display, such as points nearest a best-fit curve.
Introduction
Dynamic queries and starfield displays are tools used to access database information [1,2]. Working in conjunction, they are a convincing alternative to traditional text or forms based interfaces to databases. A starfield display is a scatter plot of relational data: each axis encodes some attribute of the data. Dynamic query tools are then used to manipulate the data in the scatter plot so that the user can view only those points in which she is interested. Starfield displays have been implemented for use in multiple dimensions, but the most common applications tend only to use two.
A major problem in the use of two-dimensional starfield displays is that of occlusion.
Simply stated, occlusion occurs in a starfield when any two data points overlap in some way: the fact that there is more than one point in the region of overlap gets obscured.
Whether occlusion occurs in a display is not necessarily related to the
attribute values of the data,
and can therefore not be affected by traditional dynamic query tools.
When there is substantial occlusion, a visualization becomes quite difficult to interpret; the influence of the occluded points cannot be perceived.
This paper will explore various options for making the presence of occluded data clear to the user.
Finally, occlusion is not the only property of a display that is unrelated to the attribute values.
This paper will explore how the techniques that have been developed to observe
occlusion can be used to represent other data; such data include (but are
not limited to) indications of outliers in the data, points nearest a best-fit
curve, and filters applied to the data. Any information that
is not an explicit attribute of the data is referred to as meta-information.
Related Work
The problem of occlusion has been studied extensively in the context of three-dimensional visualizations.
In that context, occlusion traditionally occurs because there are data in the
physical space behind the object the user is viewing.
This should be clearly distinguished from the issues discussed in this paper,
where points are actually located in the same physical space.
For example, when viewing a three-dimensional model of a molecule,
one atom might be obscured behind another.
One approach that is taken to resolve the problem of occlusion in three dimensions is "semitransparency". This approach causes the obscuring object to be made translucent: the occluded object can then be seen "through" it. There has been a good deal of work in this area [7,8].
Another approach is taken in the SDM system [3]. Here, the user is encouraged to manipulate the data to discover all of its properties, not just occlusion. One of the discoveries that can be made is that there are occluded data.
In two dimensional starfields, the most widely applied approach is called "jitter". Jitter involves the scattering of overlapping data across a wider area. The greater the "jitter" is, the greater the area across which the data are scattered. Jitter is the technique used in the commercial visualization tool Spotfire.[10]
Techniques such as excentric labeling [4] can also give a rough idea of the amount of occlusion in a given part of a two dimensional space. Excentric labeling transforms the cursor into a circle of arbitrary width; the labels associated with all of the data points located within the circle are visible. The ability to see the complete listing of labels in their context on the starfield gives a rough idea of how many points are located in the same space. One can imagine other labeling techniques, such as those described in [6], being used as well.
Finally, it is possible to view occluded information in a three dimensional graph. If the user rotates it, the occluded data become visible.
Motivation
Difficulties and Weaknesses of Existing Techniques
The existing techniques for dealing with occlusion in three dimensions are not well suited for two-dimensional data. This is to be expected: solutions designed for three dimensions will not necessarily work well in two.
Semitransparency is not well suited for dealing with data in two dimensions. This technique allows users to view only one item which is currently obscured by another.
However, in a two dimensional visualization, many points may be located in the same space, and the ability to view only one additional point may not be of any use.
Furthermore, implementations of semitransparency focus on the items directly under the cursor; this makes it difficult to get an overview of occlusion across the entire starfield.
The approach taken in SDM scales equally poorly. It may be possible to manipulate individual data points on a small scale. However, if there are several thousand points in the same space, moving them out of the way one at a time becomes a laughable prospect.
Jitter also has problems with scaling. When a small number of elements are jittered, it is easy to see the quantity of occluded items. However, once a certain degree of occlusion is reached, it becomes rather difficult to see how many points are being jittered. A few hundred and a few thousand may be nearly identical. For example, in figure 1, the top set of data points (in yellow) has three times as many elements as the bottom one (in blue).
There are other problems with jitter. The spatial positioning of data is an important indicator of attribute value. Even a slight change in the position of a data item may have great meaning. The jitter technique causes an incorrect position coding, which could have grave consequences to the interpretation of data. Finally, data points may be jittered onto other data points, thereby obscuring them. An example of both of these phenomena is in figure 2, where two points with heavy occlusion have been jittered. It is very difficult to tell which physical point belongs at which point in space.
Excentric labeling is a very powerful cursor transformation that can reveal the labels in any data, whether they are occluded or not. This can make it useful in detecting occlusion: a point that has many labels associated with it is obviously heavily occluded. However, when specifically looking for occlusion, Excentric labeling leaves much to be desired. For example, if users are trying to find the most occluded area in a scatter plot, they would first have to make the cursor as small as possible. Then they would have to move the cursor all around the display. Finally, it is necessary to count the number of labels that come up at each point. This would be a lengthy and awkward procedure. These problems are common to most location probing techniques.
Other generic labeling techniques have other problems. For example, simply placing a label that indicates the number of occluded data items over a point would have several negative effects. First, it would obscure the point itself so that other retinal properties could not be used. Second, other labels and other points may interfere with the viewer's ability to see a label devoted to occlusion clearly. These drawbacks make the use of labels to encode occlusion somewhat less powerful than one would hope.
Another approach to this problem is to use a third dimension to encode occlusion. To discover what points are occluded, the user can simply rotate the view. One important drawback to this is that it works best when there are actually data hidden in a third dimension; the hidden points are frequently located in the same physical space as each other. In this case, rotation will have no effect: the hidden data will not reveal themselves. Furthermore, if the occluded data are in a third dimension, and the space is rotated so that the users see them, then you lose the original perspective. The user may have been using the original perspective to draw some important conclusions. It is therefore important to be able to see the occluded data in two dimensions, without the use of the additional dimension.
New Approaches
Use of Traditional Retinal Properties
It is possible to solve the problem of occluded data in a traditional fashion, using existing information visualization techniques for starfields. Retinal attributes such as size, color and shape (described in [9]) can be mapped to individual data points to demonstrate the presence or absence of occlusion. For example, designers could use different colors to indicate increasing amounts of hidden data. Red could indicate five points in the same space, blue could indicate ten, and so forth.
One argument against this approach is that these properties are typically reserved for primary attributes of the data. For example, in a database plotting information about college student grades on the X-axis and SAT scores on the Y-axis, color might be used to indicate high school of origin, and shape might be used to encode gender. By using color or shape to indicate occlusion, there is one fewer property that can be used to encode attribute information.
This argument loses some of its strength in a dynamic system. If the user wishes to view an attribute using a traditional retinal property, they only have to ask for it: it should, ideally, be easy to switch between using a property to indicate an attribute and using it to represent the degree of occlusion.
There are compelling arguments for the use of traditional retinal properties for this purpose. It is possible that the division between attributes and meta-information, such as occlusion, might be lost on the user. The user may reason that if we can use color to encode ordinary attributes, why should we not be able to use it to encode meta-information. We will discuss this in detail later.
There are potential benefits to using these traditional methods. Color, shape and size all are very easily identified: one can tell instantly whether a color is green or blue, and whether a shape is square or circular. However, one definite drawback is the inability of these methods to encode ranges. Unless a very clear key is made available, it would be very difficult to tell the degree to which an item is occluded. For example, if shape were used, a circle might indicate five hidden items, and a square might indicate ten. Without referring to a key, it would be difficult for a user (or, indeed, a programmer) to remember which shape indicated a greater degree of occlusion. This is because there is no intuitive ordering offered by shape. The same is true of several other retinal properties. The first prototype we created used colors to indicate degree of occlusion: it was difficult to tell what the individual colors indicated, because there is no intuitive ordering between (say) green and blue.
The obvious fix for this is to use retinal properties that do have an intuitive ordering. For example, size coding is one property that could be used. The larger the point is, the more information could be occluded behind it. This suffers from some of the same difficulties as jitter, however. For example, it may get so big that it starts to obscure points that are not located in the same space. For an example of this, see the discussion of jitter above.
Instead of size, we might try color-coding the points so that a range of colors is used, from (for example) red to blue. There is a clear ordering implicit in this scheme; the ordering starts at red, and becomes progressively more blue. The only definitions that need to be made in this scheme is whether red means more occluded or less occluded. If the designers decided red means the point is hiding a great deal of data, then they have their encoding: the more occluded a point is, the greater the amount of red in its color. This approach does not have the limitations of the size coding: no other points will be obscured by a color change. An example of this is given in figure 3. Texture can be treated similarly.
One property that is not frequently used in starfield systems to encode attributes is orientation. A greater degree of rotation could indicate a larger degree of occlusion. The problem with this approach becomes apparent when shapes are used which are the same when rotated. For example, a circle cannot be rotated so that a difference is noticeable. Another example is a square, which can only be rotated up to 90 degrees.
Using positional data to encode occlusion is equivalent to using jitter: some of the problems with jitter are detailed above.
Use of Secondary Point Properties
The focus of our examination of the use of traditional retinal properties required that we change the property for the entire point. However, it is not necessary to change the entire point to indicate a property of the data; instead, we can change a small part of the point, and use the rest to encode whatever attribute we see fit. This allows us to avoid the problem encountered when we wish to use a traditional retinal property to encode an attribute of the data.
The first approach we took for this was to change the border around the data. We used color to indicate occlusion, just as we did with the traditional properties. We changed the color of the border instead of changing the entire point. As in our experiments with color above, we found that a range of colors encoded the information more effectively than a random color chosen for each degree of occlusion. In figure 4, an example of this sort of encoding is given.
Once the decision to change the retinal properties of the border has been taken,
it is possible to experiment in various ways with the nature of the border.
For example, if we change the shape and color of the border,
it would look as if the point was a different shape,
but had a point of the original shape and a different color marked out within it.
An example of this is figure 5: the original point was a square,
but we have changed the border to a yellow circle.
If we make the shape a circle, and use the size of the circle to encode
the degree of occlusion, then a "halo effect" is created around the point.
A larger or brighter halo corresponds to a greater degree of occlusion.
Use of Animation
If it is necessary to reserve the traditional retinal properties for the attributes of an individual data point, and it is not desirable to use the secondary properties, then it is also possible to use other indicators to encode occlusion. The design we tried was the use of animation. There are several ways to apply animation to encode occlusion.
The first implementation used a method we called "blink". This was straightforward: the more often a data point blinked, the more occlusion there was at that point. A blink is implemented by simply replacing the colored dot with a dot of the same color as the background. This potentially blots out the background grid lines, but that was not found to be a problem. After all of the points are done blinking, the process repeats from the beginning.
There was one additional decision that was made that is worth mentioning. Not all of the points blink: the ones with little or no occlusion do not. When designing the system, it was clear immediately from a qualitative viewpoint that it would be inadvisable to make the points with the least occlusion blink: it was quite aesthetically displeasing.
The second implementation was similar to the blinking technique. Instead of making the frequency of the flashes the revealing factor, greater speed showed greater occlusion. All points continually flash: those that flash the fastest are the ones with the most occlusion.
This was referred to as the "twinkle" technique.
Because all points would be flashing all of the time, it was even more important here than in the blink technique that the points with the least occlusion not flash.
Without this adjustment, the continual motion of the screen became distracting
to users.
Another approach that uses animation is the concept of spinning a data point. A point would be animated so that it appeared to spin. The speed of spinning or the length of time that it spun could be used to encode the degree of occlusion. One of the problems with this method is similar to the problem encountered when using orientation: many figures, notably the circle, cannot be effectively spun.
The Setup and Experimental Results
Implementation
We implemented these extensions to the starfield methodology on top of StarDOM[5],
a starfield system designed in the summer of 1999 by Laurent Cailleteau at the University of Maryland.
The system was written in Java,
and followed the usual pattern of such software;
there was a starfield whose contents could be adjusted by query sliders located in a side panel.
We added an additional panel,
which contained radio buttons to pick which encoding should be used for occlusion.
Figure 6 shows the final appearance of the panel.
You will note two sliders in the figure.
The top, which is labeled "Animation Speed",
controls the speed at which the animation occurs.
The bottom, labeled "Halo Size", is only used for the halo marker.
Using this control, the size of all halos can be grown or shrunk.
Enlarging them can allow for easier comparison,
and shrinking them can be useful if the view finds that they are getting
"in the way".
The major obstacle in implementing a solution for these problems was to decide what color corresponded to what degree of occlusion.
To do this, we followed a relatively simple procedure.
The algorithm started out with the assumption that the largest number of points being occluded in a single place on the starfield was the same as the number of possible colors.
Each color would get an entry in a table; each entry in the table therefore corresponded to a degree of occlusion.
The algorithm would then look at the amount of occlusion at each point on the scatter plot.
If it found that there was more occlusion at any given point than that which the table would allow for, it would combine consecutive entries in the table until that degree of occlusion could be handled.
Here is an example. Assume it starts with four colors.
It begins with the assumption that it has at most three data points hidden at any point in the picture.
The first color corresponds to no data points hidden, the second to one point hidden, the third to two points hidden, and the fourth to three points hidden.
Now assume that it finds a point on the scatter plot at which there are 6 occluded data points. It would change the table so that the first color corresponded to up to one point hidden, the second up to three points hidden, the third up to five points, and the fourth up to seven points. Now that point on the graph is accounted for by the fourth color. Since the tables may change, the points are not drawn until we have found the maximum amount of occlusion at any point in the visible data.
The only difficulty with this approach is that the influence of outliers might be too strong. For example, if one point has six thousand data items, and all of the other points have between one and one hundred items, then it is likely that the first one hundred will be one color, and the outlier will be another. This means that the user will not be able to differentiate between the points with between one and one hundred data items, unless the point with six thousand items is hidden.
One obstacle to a clear understanding of the data was that changing the range of the sliders resulted in an inaccurate reflection of occlusion information.
Specifically, the starfield could adjust itself to whatever data points were still on screen, but the assignment of colors to levels of occlusion would remain the same.
Since the maximum amount of occlusion on the screen may have changed, the old color-codings may not accurately reflect the current degrees of occlusion.
For example, everything currently on the screen may have fallen into the lowest degree of occlusion before the adjustments were made.
To account for this, the tables that determined the color codings were updated dynamically based on whatever was on the screen at that time.
That meant that there was always a range of colors values on the screen, assuming that there were varying degrees of occluded data.
Results
Having resolved these issues, the setup was ready for some tests. Three people with little experience using computers were asked to try three tasks for several of the methods implemented. The three tasks were: Find the data point with the most occlusion, find the point with the least occlusion, and find the point with the second-most occlusion. The data sets included a set of roughly 500 points, with a reasonable degree of occlusion.
Figure 3 above corresponds to the test data used.
The most occluded points, in red, have between 15 and 17 points occluded, and the least
occluded points, in blue, have fewer than 3.
The results were gathered by asking a question and then using a stopwatch to time how long it took to get an answer. Results are rounded up to the next second. The number presented is the average number of seconds it took to answer the question.
The number of people who answered the question incorrectly is supplied in parentheses.
| Jitter | Color | Border | Twinkle
|
|---|
| most occlusion | 10 (2) | 1 (0) | 2 (0) | 3 (0)
|
| least occlusion | 5 (0) | 1 (0) | 2 (0) | 3 (0)
|
| second most occlusion | 14 (2) | 3 (0) | 3 (0) | 5 (1)
|
It is difficult to draw many definite conclusions from these results.
It is clear, however, that the techniques we employed are more powerful than jitter, which takes a long time and engenders a good number of mistakes.
Twinkle was slightly more difficult than the others because of the problems in determining an individual point's blink rate when compared to the others.
Animation naturally requires more time than simply using color, which reveals itself immediately.
Ideas for Future Work
Application to Other Meta-Data Visualizations
Retinal properties in star field systems are usually applied solely to describe the primary attributes of a record in a database. Color, size or shape can be used to encode the gender of an employee, the price of a house or a student's GPA. This study has examined the use of retinal properties to encode secondary (or "meta-") attributes. The attention has been focused on occlusion. However, these techniques can be applied for other meta-attributes. For the purposes of these examples, we will assume that a colored border is being used to indicate these attributes. However, any other the methods outlined above might just as easily be used.
For example, a border color could be used to indicate outliers in the data: the farther a datum is from the majority of the data, the brighter the border might be. Position encodes this in an obvious way, but scaling may make it difficult to see if there is a significant distance from an outlier to the rest of the data. This technique might make it easier to perceive such problem data.
A related example of where our approaches might be useful is in the application of best-fit curves to scatter plot data. The closer a data point is to the curve, the brighter the border might be. This is partially encoded by using a line to draw the point, and then visually examining the points to see if they are near it or far from it. However, here again, scaling issues may make it difficult to be sure of the points that are closest to a best-fit curve. Here, having a border which indicates the closest data items may be quite useful.
Another area where these techniques might be applied is in the use of filters.
Currently, the encoding of filters forces the disappearance from the screen of all of the data points that do not meet the filter's criteria.
For example, filtering techniques such as the Magic Lens [11] remove points that do not match a given filter from an area marked out by a bounding box.
This can be thought of in another way: it is the use of the retinal property of color to encode the filter. The points are painted the background color, which effectively "hides" them. This approach causes the loss of context from the graph: it is impossible to get an overview of the rest of the data if they are all hidden. The use of context and detail is addressed by several systems; examinations of their efficacy can be found throughout information visualization literature and are beyond the scope of this paper. In this context, the approaches we have discussed above could be used to encode filtered data. For example, a black border might indicate that the points are no longer within the scope of the filters that have been applied. In this way the context can be maintained while also being able to see what points have been filtered out.
This is obviously not an optimal solution in all cases. For example, users frequently apply filters because the information field is too dense. If the points remain on the screen, this problem is not solved: none of the data points have been filtered out. However, in cases where the user wants to know which points meet certain criteria, and wants to keep an overview, this solution would be a compelling one.
This brings us neatly to another point. Using existing technology, it is simple to pick which retinal property to use to encode a given primary attribute. For example, in our system, a context sensitive menu will give the user the choice of using color, shape or size to describe an attribute of a point. To encode meta-attributes effectively, a similarly simple interface must be used to choose how you wish to encode each one in turn. This must take into account the fact that it is only possible to use one retinal property to encode one piece of information. For example, if color is being used to encode occlusion, it is necessary that it not be used to encode an attribute's value.
It is beyond the scope of this paper to discuss the best form of interface to apply retinal attributes to a starfield system. However, it seems apparent that one must unify the menu used to apply retinal properties to attributes and the one used to apply them to meta-attributes. In our starfield system, the context sensitive menu which appears when picking the retinal property to apply to an attribute would also appear when applying the property to a meta-attribute. In addition to choices for color, size and shape, it would also contain choices for blink, twinkle, or halo. This would have the added benefit that the secondary retinal properties and the animation effects also become available for the encoding of primary attributes.
This approach would blur the line between the attribute and the meta-attribute. The major difference between the two is that the attribute is domain-specific, while the meta-attribute is not. This difference may not be important to the user in certain cases; whether it does or does not it a matter for future work.
Conclusion
Encoding of occlusion could play an important role in information visualization. We have suggested techniques to address the problems involved in carrying it out. The three major approaches, traditional retinal attributes, secondary (or alternate) retinal attributes and animation, each have advantages and disadvantages. The current state of the art in information visualization, however, does not do enough to address the issue.
A potentially more important role that this work could play involves the use of these encoding techniques outside their application to occlusion. Other meta-attributes could strongly benefit from the application of these techniques. These meta-attributes include: distinguishing filtered data, indicating outliers, and indicating points closest to a best fit curve. It is very likely that further study would lead to further meta-attributes of individual points and approaches that encode these meta-attributes.
Bibliography
- Shneiderman, B.. Dynamic Queries for Visual Information Seeking. IEEE Software, 11(6), 70-77, 1994 IEEE
- Ahlberg, C., Shneiderman, B., Visual Information Seeking: Tight Coupling of Dynamic Query Filters with Starfield Displays Information Visualization, Proceedings of ACM CHI'94 Conference on Human Factors in Computing Systems 1994 v.1 p.313-317
- Chuah, M., Roth, S., Mattis, J., Kolojejchick, J., SDM: Selective Dynamic Manipulation of Visualizations. Proceedings of the ACM Symposium on User Interface Software and Technology 1995 p.61-70
- Fekete, J-D and Plaisant, C Excentric Labeling: Dynamic Neighborhood Labeling for Data Visualization. Proceedings of ACM CHI 99 Conference on Human Factors in Computing Systems 1999 v.1 p.512-519
- Cailleteau, L, Interfaces for Visualizing Multi-Valued Attributes: Design and Implementation using Starfield Displays. Partial requirement for the Diplome d'Etudes Approfondies. Final version, 1999
- Li, J., Plaisant, C., Shneiderman, B. (1998) Data Object and Label Placement for Information Abundant Visualizations Workshop on New Paradigms in Information Visualization and Manipulation (NPIV'98) In conjunction with the CIKM'98, ACM, New York.
- Zhai, S., Buxton, W., Milgram, P. The Partial-Occlusion Effect: Utilizing Semitransparency in 3D Human-Computer Interaction, ACM Transactions on Computer-Human Interaction, Vol. 3, No. 3, September 1996, Pages 254-284
- Zhai, S., Buxton, W., Milgram, P. The "Silk Cursor": Investigating Transparency for 3D Target Acquisition. Proceedings of ACM CHI'94 Conference on Human Factors in Computing Systems 1994 v.1 p.459-464
- MacKinlay, J. Automating the Design of Graphical Presentation of Relational Information. ACM Transactions on Graphics, 5(2), 111-141, 1986.
- Spotfire Pro. http://www.spotfire.com
- Fishkin, K., Stone, M. C. Enhanced Dynamic Queries via Moveable Filters. Proceedings of CHI'95 1995 415-420.
Web Accessibility