First-of-its-kind learning platform could give students who are blind or visually impaired access to STEM-related graphics

Development and evaluation of a first-of-its-kind remote learning platform providing people who are blind or visually impaired (BVI) nonvisual access to STEM-related graphical information is the focus of a $748,000 National Science Foundation grant to the University of Maine.

The project, “A Remote Multimodal Learning Environment to Increase Graphical Information Access for Blind and Visually Impaired Students,” is led by Nicholas Giudice, UMaine professor of spatial informatics who directs the Virtual Environment and Multimodal Interaction (VEMI) Laboratory. The research team also includes Justin Dimmel, UMaine assistant professor of mathematics education and instructional technology, and Stacy Doore, visiting assistant professor of computer science at Bowdoin College.

The system uses combinations of nonvisual inputs, such as vibration, speech and auditory information, that allow BVI users to feel and hear the visual content of graphics as they move their hand around the touchscreen of smartphones and tablets. With graphical educational materials at the core of all STEM disciplines, this information access is critical for improving BVI students’ classroom outcomes.

The project will conduct one of the largest experiments ever performed on graphical access with BVI participants, with results leading to the development of a robust and economically viable solution for the BVI community.

The research directly addresses the nationwide lack of accessible STEM resources, with project outcomes having significant potential for improving both educational opportunities and workforce training for many BVI individuals, Giudice says. Broader impacts of the information-access technology developed in this project could also benefit older adults experiencing vision loss or sighted people in “eyes-free” applications, such as operating car infotainment systems when driving.

This newest award is the latest in six years of research by Giudice and the VEMI Lab focused on using commercial smartphones and tablets to provide BVI people with nonvisual and multimodal access to visual graphics. The work has been supported by two other NSF grants and has been published in over 30 papers and conference presentations.

The research has led to the development of a new class of information access technology that has gained significant national attention and has potential for solving the long-standing graphics access problem for people with visual impairment. By creating an accessible remote learning platform that can work on a BVI user’s personal smartdevice, this most recent project takes the research outside of the lab and directly to the people who can most benefit from its application.

Contact: Margaret Nagle, 207.581.3745