Multimodal MATLAB: data “visualization” for the blind.

James A. Ferwerda and Victor Tsun-Hay Kwok

Data visualization methods have revolutionized the fields of Science, Mathematics, Engineering and Medicine. The ability to create, view, and interact with meaningful visual representations of complex multidimensional datasets has been a powerful aid to discovery, innovation, and teaching in all these fields. Unfortunately the graphical tools that have been of such benefit to these communities are largely inaccessible to its blind and visually impaired members. To address this issue we have developed a new set of data “visualization” tools for people with visual disabilities. The tools extend the graphics capabilities of MATLAB, the widely-used open-source technical computing environment. We call the tools MIST (for Matlab Interactive Sonification Toolbox). MIST supports the creation of multimodal graphs that include visual, auditory, and synthetic speech components. Users can interact with these multimodal graphs using a mouse, tablet, or keyboard commands. A key innovation is the ability to create multidimensional MIDI “soundmaps” that can be used to represent complex data attributes in sonified form. We are currently evaluating the usability and effectiveness of MIST with both blind and sighted users. We are also extending the toolkit to include tactile data representations and support novel non-visual interfaces.

This work was supported by NSF grant IIS-0113310 and the Cornell Program of Computer Graphics.