A Data Capture and Gesture Recognition System to Enable Human-Robot Collaboration; Exploring the Ethics and Accountability of AI-Driven Decision-Making in Modern Combat

How do we design autonomous systems that have the capability of performing effectively and ethically alongside or as a replacement for human beings in military scenarios? This is the question that has many governments and companies stumped, and a question both that my technical capstone and STS thesis are attempting to help answer. My technical research worked towards improving human-robot interactions in search and rescue scenarios through the creation of a new gesture recognition system for the US Army, while my STS thesis examined the ethical and accountability concerns that have emerged due to the addition of autonomous weapons in modern warfare. Whereas one project aims to improve human’s capability to save lives and the other delves into autonomous systems that can take them, both projects help understand what occurs when human decision-making is altered or removed completely from mission-critical operations. Together these two projects help shed light on the reality of autonomous systems in our modern society, and the need for accountability measures, openness, and ethical guardrails when developing and implementing these new systems.

My technical capstone project team created a new gesture recognition system that aims to enhance human-robot collaboration in challenging environments with a focus on military search and rescue operations. We created a near real-time pipeline based on a vision-language model informed by retrieval-augmented generation and chain-of-thought prompting that could classify 7 upper-body military gestures. These 7 gestures were selected for their simplicity and relevance in noisy or hazardous environments where spoken communication is often impractical or impossible. The system captures RGB-D video, selects key frames, and identifies the gestures before sending unique commands based on the identified gesture to a robot using a client-server architecture. In our trials the system achieved around 80% accuracy for correctly identifying one of the 7 gestures. This work contributes to creating a more reliable means of human-robot communication in military search and rescue operations.

For my STS research I analyzed the ethical issues that come with lethal autonomous weapon systems in the battlefield and how these challenge the moral principles of just war theory. My analysis focused on two specific criteria of just war theory, distinction and proportionality. I argued that fully autonomous systems cannot reliably differentiate between soldiers and civilians especially in unideal conditions or correctly assess the proportionality of an attack in a meaningful way. These flaws are further exacerbated by their manufacturer's black box development approach and the clear absence of accountability systems for these machines, which creates a difficulty in assigning blame or responsibility in the event of a failure. My work provided an analysis that can be used by government agencies and companies to to conclude that fully autonomous weapon systems, at least in their current state, should not be allowed onto current battlefields.

Evaluating modern technical systems requires understanding both their operational capabilities as well as the implications they have on the people and places they affect. Through both my technical and STS projects I've learned that being a “good” engineer doesn’t just lie in making systems work from a technical standpoint, but in ensuring those systems are ethical and accountable. As computers and autonomous systems are increasingly being asked to infer and even act upon their analysis of human intention and decision making, it is necessary that engineers design future systems with an STS perspective in mind.

School of Engineering and Applied Science

Bachelor of Science in Sytems Engineering

Technical Advisor: Tariq Iqbal

STS Advisor: William Davis

Technical Team Members: Sarah Naidu, Evan Smith, Camp Hagood, Aramis Rolly, Sujan Sarker, Cory Hayes, Tariq Iqbal