Skip to main content


Summary of HCIL Projects in Temporal Visualizations

Quick Links: LifeLines2, LifeLines, Similan, LifeFlow,
EventFlow, PatternFinder, PatternFinder in Amalga, TimeSearcher 1-3,
Learning Historian, LifeLines (original)


EventFlow : Exploring Point and Interval Event Patterns

The HCIL's ongoing work with temporal event records has produced powerful tools for analyzing and exploring patterns of point-based events (Lifelines2, LifeFlow). However, users found that point-based events limited their capacity to solve problems that had inherently interval attributes, for example, the 3-month interval during which patients took a medication. To address this issue, EventFlow extends its predecessors to support both point-based and interval-based events. Interval-based events represent a fundamental increase in complexity at every level of the application, from the input and data structure to the eventual questions that a user might ask of the data. Our goal was to accomplish this integration in a way that appeared to users as a simple and intuitive extension of the original LifeFlow tool. With EventFlow, we present novel solutions for displaying interval events, simplifying their visual impact, and incorporating them into meaningful queries.

LifeFlow : Visualizing an Overview of Event Sequences

Event sequence analysis is an important task in many domains: medical researchers study the patterns of transfers within the hospital for quality control; transportation experts study accident response logs to identify best practices. In most cases they deal with more than thousands of records. While previous research has focused on searching and browsing, overview tasks are often overlooked. We introduce a novel interactive visual overview of event sequences called LifeFlow. LifeFlow scales to any number of records, summarizes all possible sequences, and highlights the temporal spacing of the events within sequences.

Similan : Finding Similar Temporal Categorical Records

Electronics Health Records (EHRs) are being collected by leading health organizations. These EHRs contains millions of records with patient histories. Challenges arise when a practitioner would like to to find records of patients with similar symptoms to the targeted patient in order to guide the treatment of the target patient. Finding similar patients from millions of records with patient histories is a challenging problem. The initial goal of this project is to enable discovery and exploration of similar records in temporal categorical dataset. The main challenge is how to define "similarity".

We are designing a customizable similarity measure that is flexible enough to capture different definitions of similarity according to users' need and allow them to customize this measure in their own ways. We build a prototype tool called Similan, an interactive tool for finding similar records from temporal categorical data. Similan allows users to specify a target record, customize parameters in the similarity measure computation and provides visualization techniques to help users understand and explore the search results. We also extend Similan to become a query-by-example tool (searching by giving an example and rank-by-similarity). Although this project was first motivated by EHRs, applications of Similan and the similarity measure are not limited to the medical domain.

LifeLines 2 : Discovering Temporal Categorical Patterns Across Multiple Records

Electronic health records (EHRs) contain a wealth of information. Categorical event data such as complaints, diagnoses, treatments, etc., are important, and play important roles in health providers decision making. However, past research efforts have been focused on numerical data and single-record visualization techniques. Discovering patterns of categorical events across multiple records are supported in limited ways.

Lifelines2 is an interactive visualization tool for visualizing temporal categorical data across multiple records. The goal of the project is to enable discovery and exploration of patterns across these records to support hypothesis generation, and finding cause-and-effect relationships in a population. These tasks are not specific to the medical domain, but we were first motivated by EHRs.

PatternFinder in AMALGA : Temporal Query Formulation and Result Visualization in Action

The core ideas of PatternFinder has been implemented in real medical systems. Through the collaboration with Washington Hospital Center, we have integrated both the sophisticated query specification for temporal patterns and the ball-and-chain result visualization with their production system Amalga (previously called Azyxxi). The query forms have adopted the look and feel of the existing Amalga interface to help physicians and medical personnel to adapt. The query form has been simplified to not overwhelm the users. The top screenshot shows patients with Hemoglobin level increasing by 50% within 10 days then decreasing 30% within 10 days. Results are presented both in a table view (bottom) and in the visual overview (to the right). For each patient all events that match the query are summarized, and users can expand to see more details about all combinations of matching events. The three filters have green, purple, and blue circles, which show up in the results display on the left side, indicating a match for a filter.

PatternFinder : Query Support for Electronic Health Records

Pattern Finder is an integrated interface for visual query and result-set visualization for search and discovery of temporal patterns within multivariate and categorical data sets. We extend work on visual query languages for temporal data with a novel interface and address the presentation of the result sets. Temporal patterns are sequences of events with inter-event time spans. Pattern queries allow events, event sets, event attributes, and time spans to be specified so as to produce powerful queries that are difficult to express in other formalisms. Finding patterns of events over time is important in searching patient histories, web logs, news stories, or in tracking criminal activities.

LifeLines : Visualizing Patient Records

Computerized medical records pose tremendous problems to system developers. Infrastructure and privacy issues need to be resolved before physicians can even start using the records. Non-intrusive hardware is required for physicians to do their work (e.g. interview patients) away from their desk. But all the efforts to solve these problems will only succeed if appropriate attention is also given to the user interface design. Long lists to scroll, clumsy searches, endless menus and lengthy dialogs will lead to user rejection. But techniques are being developed to summarize, filter and present large amounts of information, leading us to believe that rapid access to needed data is possible with careful design.

In our past project for the Maryland Department of Juvenile Services we have developed a new technique called Life-Lines to visualize personal history records. We are now working with IBM Watson Research Center to extend the technique to medical records. LifeLines provides a general visualization environment for personal histories. A one screen overview of the record using timelines provides direct access to the data. For a patient record, medical problems, hospitalization and medications can be represented as horizontal lines, while icons represent discrete events such as physician consultations, progress notes or tests. Line color and thickness can illustrate relationships or significance. Rescaling tools and filters allow users to focus on part of the information, revealing more details.

TimeSearcher (1-3) : Visual Exploration of Time-Series Data

Widespread interest in discovering features and trends in time- series has generated a need for tools that support interactive exploration. We have built a prototype environment for interactive querying and exploration of time-series data. Queries are built using timeboxes: a powerful graphical, direct-manipulation metaphor for the specification of queries over time-series datasets. These timeboxes support interactive formulation and modification of queries, thus speeding the process of exploring time-series data sets and guiding data mining. The prototype includes windows for timebox queries, individual time-series, and details-on-demand. Other features include drag-and-drop support for query-by-example and graphical envelopes for displaying the extent of the entire data set and result set from a given query.

Learning Historian

While simulations seem to be useful, we still need to understand how these environments can be designed to effectively promote student learning. The aim of the Learning Historian is to provide a richer environment to learners while they freely explore the behavior of a simulation. The basis of the Learning Historian is to record the history of the interaction with the simulation and allow this history to be replayed for review, sent with a message, used in a tutorial, or replayed as a series of variants to facilitate comparisons and explorations.

Lifelines (Original) : User Interfaces for Juvenile Justice Information Systems

We have been working with the Maryland Department of Juvenile Services to design advanced user interfaces for an information system used by 600 employees. DJS is responsible for handling juvenile case referral for the entire state of Maryland. Our first step was to observe users, inteview users and managers, and survey 300 users with the Questionnaire for User Interface Satisfaction. We have refined and applied our user interface re-engineering methods to provide guidance for short-term improvements to the current system and are now turning to longer range recommendations. Our proposed interfaces use dynamic queries and starfield displays to the reveal trends and anomalies in case statistics. We are developing new techniques to visualize youth records and personal histories by showing multiple time lines with selectable markers to retrieve detailed information. For managers, we are proposing an organizational visualization tool that will allow effective navigation of large human services databases.

Related Workshops from HCIL

Visualizing Personal Histories: a Workshop (July 21-22, 1997)

Personal Medical Devices Workshop: Increasing Patient Healthcare Participation (June 3, 2004)

Interactive Visual Exploration of Electronic Health Records (May 30, 2008)