Automatic Captioning Software from any Audio Source; Deaf Disparity: How Deaf People Use Technology that Expects them to Hear

Clark, Edward, School of Engineering and Applied Science, University of Virginia
Norton, Peter, EN-Engineering and Society, University of Virginia
Feng, Lu, EN-Comp Science Dept, University of Virginia
Cohoon, Jim, EN-Comp Science Dept, University of Virginia

Members of the Deaf and Hard-of-Hearing (DHH) community face hardship due to inadequate accessibility accommodations in modern devices and interfaces. How can the quality of life for deaf and hard-of-hearing people be improved?

To diminish the barriers that can impede DHH individuals’ careers in a virtual environment, I am proposing to develop automatic captioning software. Using natural language processing, the application acts as an overlay that will caption any computer audio that is playing on the user’s machine (Zoom Meetings, YouTube videos, etc.). Downloading specific add-ons for each platform are unnecessary as the software will attempt to caption any audio that is currently playing. With this tool, DHH individuals can seamlessly read speech said on their computer without assistance or extra devices.

The DHH community often must use interfaces that expect users to hear, adapting them to their needs. Current accessible interfaces either work poorly or are quickly outdated as technology continues to advance. Remote work has become ubiquitous, and DHH persons need adequate tools to engage in such work. To support their autonomy, DHH individuals require technology that serves them by default, as current accommodations are inadequate and separate assistive tools cannot keep pace with current innovation

BS (Bachelor of Science)
deaf accessibility, realtime captioning, natural language processing, speech to text

School of Engineering and Applied Science

Bachelor of Science in Computer Science

Technical Advisor: Lu Feng

STS Advisor: Peter Norton

All rights reserved (no additional license for public reuse)
Issued Date: