Demo video (AVI (2.3Mb), MOV (4.8Mb))

Multimodal MATLAB: data “visualization” for the blind

Data visualization methods have revolutionized the fields of Science, Mathematics, Engineering and Medicine. The ability to create, view, and interact with meaningful visual representations of complex multidimensional datasets has been a powerful aid to discovery, innovation, and teaching in all these fields. Unfortunately the graphical tools that have been of such benefit to these communities are largely inaccessible to its blind and visually impaired members. To address this issue we have developed a new set of data “visualization” tools for people with visual disabilities. The tools extend the graphics capabilities of MATLAB, the widely-used open-source technical computing environment. We call the tools MIST (for Matlab Interactive Sonification Toolbox). MIST supports the creation of multimodal graphs that include visual, auditory, and synthetic speech components. Users can interact with these multimodal graphs using a mouse, tablet, or keyboard commands. A key innovation is the ability to create multidimensional MIDI “soundmaps” that can be used to represent complex data attributes in sonified form. We are currently evaluating the usability and effectiveness of MIST with both blind and sighted users. We are also extending the toolkit to include tactile data representations and support novel non-visual interfaces.

For more information contact:

James A. Ferwerda

Program of Computer Graphics

580 Rhodes Hall

Cornell University

Ithaca, NY 14853

607-255-7365

607-255-0806 (fax)

jaf@graphics.cornell.edu

http://www.graphics.cornell.edu/~jaf

Credits:

• Program development and coding: Victor Tsun-Hay Kwok
• Concept and design: James A. Ferwerda

Acknowledgements:

• National Science Foundation, ITR/IIS-0113310
• Cornell Program of Computer Graphics