Gesture Controlled Robotic Vehicle; Analysis of Learning Process of Augmentative and Alternative Communication Devices

Communication is a vital ability that is many times taken for granted. My research in STS was a deep dive into the sociotechnical system of learning to use augmentative and alternative communication (AAC) devices. In my research I learned about the difficulties that verbally challenged people and their families face as they learn to use communication devices. My technical project was an exploration into intuitive design where I remapped the control of a robotic vehicle from the more typical controllers to a glove that captures user gestures to drive the vehicle. My work on both projects allowed me to gain a better understanding of the importance of clear communication between teams.

The technical project was a gesture driven robotic vehicle. This is intended to be a toy for children around 11 to 15 years old. It is common to see remote controlled cars marketed towards children this age with a remote shaped like a video game controller. The goal of the project was to take the typical controller redesign it to be more intuitive to use. Instead of using joysticks and buttons, our glove-based control uses the user’s gestures to control the car. The glove has a gyroscope sensor to capture the basic movements of the hand: tilt forward, backward, left, and right. The tilting of the hand is converted into a combination of the vehicle’s movement: forward, backward, strafing left and right, and turning left and right. The vehicle has an analog transmission camera that can be viewed through an external monitor. The vehicle also has an ultrasonic sensor that is used to give haptic feedback to the user about the vehicle’s surroundings. When the car approaches an obstacle with the ultrasonic sensor, a haptic feedback motor in the finger of the glove vibrates to warn the user.

My STS research centers around better understanding and characterizing the process of learning to use AAC devices. By viewing the problem through the context of the social model of disability, I found that the process can be modeled using actor-network theory. After identifying some of the important actors of the network, I gathered statements from users and their families about the experiences in learning to use AAC devices. Through analyzing the research, I was able to come to the conclusion that the actors of the network do not communicate and cooperate for the benefit of the user.

Upon first glance, these two projects are very disjoint from one another. However, as I continued to work and learn on both projects, I came to understand the importance of clear communication. Throughout the process of designing and constructing my technical project, I had to continuously consult with my team members and mentors to iteratively improve the design. Since the design was meant for children, we had many discussions about how it could be made safer and more durable to withstand rough use. Through these discussions, I was able to put myself in the shoes of the designers of these AAC devices. The designers need to balance creating a design specific enough for each individual user as well as general enough that they can be distributed widely. The improvements to these designs need to be driven by the feedback of the users and medical personnel. The cycle of feedback and improvement I experienced in my project helped me understand the problems in the cycle of feedback of AAC devices.

School of Engineering and Applied Science
Bachelor of Science in Computer Engineering
Technical Advisor: Adam Barnes
STS Advisor: Kathryn Neeley
Technical Team Members: Ruhul Quddus, Nima Razavi, Ian Le, Kenny Zhang