Harmony In Isolation: A Real-Time Vocal Harmonizer; Division of Responsibilities for the Security of Voiceprint Biometric Collection and Use

A voice is more than a tool of expression and personality, because all the unique characteristics of a voice make up a voiceprint, an identifiable, trackable biometric that deserves protection. As the use of biometric authentication and voice recognition becomes more widespread, the security of our personal identifiers, such as our voiceprint, must be taken into consideration. Beginning from the engineer’s role in this consideration, the technical project aims to create a vocal harmonizer, a device to assist musicians and singers in times of distance learning by applying hardware and software to make one voice sound like many. On the other end of the spectrum, the science, technology, and society (STS) focus will explore the data privacy legislation that regulates the collection and use of an individual’s voiceprint in technologies ranging from banking to voice-activated assistants. The tightly coupled technical and STS topics describe the process of developing a voiceprint technology, along with the perspectives of the various social groups who influence the collection, use, storage, and regulation of voiceprint biometrics in technology.
Originally inspired by the social distancing requirements due to COVID-19 restrictions, the technical report enumerates the design and development of a device to allow musicians to create harmonies without additional people. The proposed device, a real-time vocal harmonizer, allowed the user to sing or play an instrument into a microphone while simultaneously pressing down a chord on a keyboard, and then produced the output of the voice or instrument reharmonized to the chord that was played. This device was implemented using a custom printed circuit board (PCB), a MyRIO processor for real-time processing, and a 3D-printed attachment to secure the electronics to the keyboard.
In the proposal for the vocal harmonizer, three deliverables were outlined for evaluation of the final product: (1) The output responds to changes on the inputs in real time, (2) the timbre of the output resembles the timbre of the microphone input, and (3) the pitches of the output match the keys pressed on the keyboard. The final product met all proposed deliverables. In addition, potential security risks posed by the creation of a voice modifier were addressed limiting the data collection to include only the fundamental frequency and harmonics of the user’s voice and reducing the storage time of voice data.
The inspiration for researching the influencing network for voiceprint biometric technologies started as an initial curiosity about how biometric data is used and protected. The STS discussion explored the existing regulations and current practices of voice data collection, consent, and use practices, and considered how to improve the ethical considerations in the development and regulation of devices capable of collecting, storing, and using voice and biometric data. Latour and Callon’s Actor Network theory approach was used to create a framework analyzing the interactions between the social groups involved in the construction of voiceprint technologies. The actor network was developed through a collection of market trends, personal observations, law reviews, and journal articles discussing current data practices.
The STS topic was narrowed to exploring missing interactions found in the actor network and methods to balance responsibility for consumer biometric privacy between engineers, government, users, and businesses. National data privacy legislation was posed as a solution to define the consent requirements and level of care for collected biometric data. To support this solution, the STS topic recommended actively incorporating into the engineering design process the professional judgement of engineers, protections provided by the legal system, and the participation of consumers. Because the vocal harmonizer involves a subset of the user’s voiceprint, the design process used in the technical project was included in the STS discussion as a type of case study of the engineer’s role in the proposed engineering design process.
Due to the sensitive nature of biometric data, there is a clear need for regulation to expand user privacy protections to include biometric data. To better protect consumer’s biometric privacy, one solution would be national legislation and self-regulation encouraged through a modified design paradigm. Because we have no control over mal actors against consumer data privacy, we can focus our efforts on reducing the extent of biometric data collected and the storage time in addition to the encryption in place on stored data.

School of Engineering and Applied Science
Bachelor of Science in Electrical Engineering
Technical Advisor: Harry Powell
STS Advisor: Catherine Baritaud
Technical Team Members: Nate Hunter, Noah Mills