Skip to main content


Zhao, H. (April 2006)
Interactive Sonificaton of Abstract Data - Framework, Design Space, Evaluation, and User Tool
Ph.D. Dissertation from the Department of Computer Science

For people with visual impairments, sound is an important information channel. The traditional accommodation for visually impaired users to access data is to rely on screen readers to speak the data in tabular forms. While speech can accurately describe information, such data presentation tends to be long and hard to realize complex information. This is particularly true in exploratory data analysis in which users often need to examine the data from different aspects. Sonification, the use of non-speech sound, has shown to help data comprehension. Previous data sonifications focus on data to sound attribute mapping and typically lack support for task-oriented data interaction. This dissertation makes four contributions. (1) An Action-by-Design-Component (ADC) framework guides auditory interface designs for exploratory data analysis. The framework characterizes data interaction in the auditory mode as a set of Auditory Information Seeking Actions (AISA). It also discusses design considerations for a set of Design Components to support AISAs, contrasted with actions in visualizations. (2) Applying the framework to geo-referenced statistical data, I explore its design space. Through user evaluations, effective design options were identified and insights were obtained regarding human ability to perceive complex information, especially those with spatial structures, from interactive sounds. (3) A tool, iSonic, was developed, with synchronized visual and auditory displays. Forty-two hours of case studies with seven blind users show that iSonic enables them to effectively explore data in highly coordinated map and table views without special devices, to find facts and discover data trends even in unfamiliar geographical contexts, Preliminary algorithms are also described to automatically generate geographical region spatial sweep orders for arbitrary maps. (4) The application to geo-referenced data demonstrated that the ACD framework provided a rich set of task-oriented actions (AISAs) the were effective for blind users to accomplish complex tasks with multiple highly coordinated data views. It also showed that some widely used techniques in visualization can adapt to the auditory mode. By applying the framework to scatterplots and line graphs, I show that the framework could be generalized and lead to the design of a unified auditory workspace for general exploratory data analysis.

MonoTrans Screenshot

More information

Tech Reports
Video Reports
Annual Symposium

Seminars + Events
HCIL Seminar Series
Annual Symposium
HCIL Service Grants
Events Archives
HCIL Conference Travel Award
Job Openings
For the Press
HCIL Overview
Become a Member
Collaborating Groups + People
Academic Visitors
Join our Mailing List
Contact Us
Visit Us
HCIL Store
Give the HCIL a Hand
HCIL T-shirts for Sale
Our Lighter Side
HCIL Memories Page
Faculty/ Staff
Ph.D. Alumni
Past Members
Research Areas
Design Process
Digital Libraries
Physical Devices
Public Access
Research Histories
Faculty Listed by Research
Project Highlights
Project Screenshots
Publications and TRs
Studying HCI
Masters in HCI
PhD in HCI
Visiting Scholars
Class Websites
Sponsor our Research
Sponsor our Annual Symposium
Active Sponsorship
Industrial Visitors