UIST 2005 Interaction Contest
Description of Training Dataset, Tasks, Metrics and Rules

The training dataset and tasks is representative of the dataset and tasks to be used in the live contest at the conference. It also specifies the format of the output log file that will be used by judges. We will post revisions of the training dataset until all your questions have been answered (Date still to be determined)

TRAINING DATASET

In the training dataset we provide two scenes

Scene #1

The first scene is a room where a set of small objects (e.g. forks, spoons, scalpel, or scissor) should be positioned on a destination flat surface (e.g. a dining / surgical table). The small objects located on a source surface should be initially located far away from the destination. Between the source and the destination, the room contains obstacles, such as columns / pillars, pieces of furniture, or people that should be avoided. The small objects on the source surface are mixed with distracter objects that should not be moved but reactive to unnecessary contact or interaction.

This example could represent an operation room with tools stored on shelves or in drawers that should be positioned next to an operating table in some specific order and location. The bed and other distracter objects can be on the way from the source table to the destination table.

It could also represent a game like “The Sims” where the dining table should be set with plates, forks, and knives originally located on the kitchen. The dining table to set being on the dining room. Tables or people can be in between the destination and the source as the distracter objects.

Scene #2

The second scene is the Earth, with specific locations marked on the textured surface. It could be generalized as a set of marks on the surface of a convex polyhedron. The size of the marks is several orders of magnitude smaller than the size of the polyhedron; pointing should be as precise as the marks on the texture are.

How different will the final dataset be?

The scenes will be different, and there might be more than two. The range of scales will be different. The scenes and objects will be different, in such a way that there is no reason or need to include shape recognition algorithms in your tool (in fact we won’t allow it). There will be many tasks, some needing high precision, some not.

The point of the contest is to compare techniques for human-computer interaction (not pattern recognition algorithms and autonomous navigation techniques that might also have useful applications in similar situations). If you have any questions in this regard please contact us. You can describe your technique to us and we will confirm if this is acceptable of not (and keep your plans confidential).

SAMPLE TASKS

For the first scene, the task consists in moving small objects from source locations to destination locations with the highest accuracy in the shortest time. The locations include position (X,Y,Z) and orientation (DX,DY,DZ) in 3D.

(Note: a precise sample training set of tasks will be posted here soon with the training dataset)

During the contest, tasks may include:

· Pointing to an object or mark, possibly at a very different scale

· Picking one or more objects, avoiding distracters

· Extracting an object out of several combined objects without contacting distracters

· Navigating through a number of obstacles

· Rotating an object to align it with a specified shape or location

· Docking objects

· Following a narrow path

METRICS

Speed and precision will be measured to compare interaction techniques. The total time to complete the tasks will be easy to measure with a stop watch during the contest, but we will also collect detail information from log files, and qualitative preference data from the audience during the event.

We will use a simple formula combining speed and accuracy information contained in the log files. If the overall interaction to perform the task takes time t and the distance between an object i and its target position is d_i, the formula is:

where t is expressed in milliseconds and d is in scene coordinates. The constants a and b will be specified for the task instances since they are very sensitive to actual coordinates and ranges.

d is an Euclidian distance between the expected target and the specified point for task 2. For task 1 where orientation is also important, d is measured as the shape distance. For each scene, we will provide the desired precision of the positions in scene coordinates.

The software used by the participants has to produce a log file containing for each object manipulated time-stamped information about the trajectory of the object and its end position. MORE DETAILS ARE GIVEN WITH THE TRAINING SET.

Please note that if too many teams apply by the June deadline we may need to do a pre-selection using results with the training dataset.

RULES

If you are not sure… ask us!

All categories of competitors (academic, commercial, government) may participate.
We especially encourage student and class submissions.
You can compete as a team, or a single individual, but only one person will operate the interface in the competition.
The contest is open to all except the contest organizers.

You can

• use any existing off the shelf commercial product or research prototype, and of course you may combine tools
• use 2D or 3D, or both, in multiple views
• use any input or output device you want, but please try to find a way to make your method visible by the audience (e.g. force feedback could be made visible on the screen as well)
• use all your hands, feet and other body parts.
• use any computer equipment you want but you have to bring your equipment to the conference by yourself, and be able to set it up on stage in only a couple of minutes (if you are not sure, ask us).

You cannot:

• change the problem to a problem of complete automation (we are looking at human-computer interaction solutions) but tools are OK, so in doubt, ASK us and describe your idea. For example sensors can warn you that you are approaching a wall, but objects can't automatically arrive at their destinations without your intervention.
• use specially crafted image processing algorithms to find the best final position (in any case remember that the competition will differ enough from the training set that most likely those algorithms would not help anyway).

Keep in mind that:

• your equipment should be used for the live contest (we can’t provide anything or pay for the shipping)
• calibration of any device must be done in advance, there will not be enough time during the live session.

NEXT? DOWNLOAD the training dataset and tasks (requires registration)

Questions? Send email to us as: Jean-Daniel.Fekete@inria.fr; plaisant@cs.umd.edu; kevin_ucd@acm.org

Return to UIST 2005 Contest