Abstract
Imagine swiping your hand through the air to skip a song, pinching fingers together to zoom on a holographic map, or typing an email in midair, all without touching anything. This vision of contact-free, human-computer interaction (HCI) has driven new research and commercial investment in gesture-control wearables that translate muscle signals, motion, and other bodily biodata into machine commands. As the world increasingly relies on computer machines for data processing, communication, automation, and entertainment, the maturation of gesture-control technology will undoubtedly lead to its diffusion across consumer markets and work sites. Moreover, the maturation process involves two semi-contradictory design drivers with (1) optimization or raw responsiveness and (2) user-centric design or accessibility, which are both meant to reduce the barrier of operation between the human body and the machine. This begs the question: how does the design of such systems affect the datafication and commodification of the body, and what happens when gesture-control wearables begin to dominate not just everyday devices, but our own work and personal lives?
My capstone technical project develops a wrist-worn, gesture-control wearable targeting high accuracy and low latency. The system fuses surface-electromyography (sEMG) sensors and an inertial measurement unit (IMU) to capture muscle-activation and motion data, respectively, that is classified by a machine-learning model to output digital computer commands. Beyond the signal processing and latency challenges, dataset bias was a major complication since muscle mass, wrist geometry, and handedness differ across bodies. Classification models trained on nonrepresentative data can systematically disadvantage bodies that deviate from the assumed norm, which is typically the majority class. And this raises both technical and ethical questions about who these systems are actually built for.
My accompanying STS research paper explores what happens as gesture-control wearables transition from consumer gadgets into workplaces and everyday life. Drawing on the surveillance capitalism, neoliberal subjectivity, and Social Construction of Technology (SCOT) frameworks, my paper argues that continuous biodata collection commodifies bodily motion while design choices favoring ‘seamlessness’ and ‘efficiency’ serve employer and platform interests over user autonomy. Together, my technical project and STS research reveals that the stakes of gesture-control wearable design go beyond the technical and asks: who defines seamless and efficient; who stands to gain the most from these design choices; whose bodies are penalized when they fail to conform; and, what vision of the always-on, productive individual is quietly encoded in every gesture the system accepts?
