Abstract
In today’s economic environment, the monetary success of large manufacturing firms (and the countries they operate in) depends heavily on how well they can compete with other firms on a global scale. This high-stakes, global competition has pushed manufacturing firms to implement highly automated processes to improve productivity and reduce costs. Furthermore, firms are now capable of automating processes to a much greater extent than ever before due to advances in AI and robotics, opening the door to unforeseen consequences. The large-scale problem that frames both my technical and STS research is that many firms are implementing such automation with little to no consideration of the harm that these new systems will have on their human workforces and their surrounding societies. Researching this problem can help provide insights necessary to improve the outcomes and benefits of automation, while also mitigating potentially catastrophic economic and employment recessions. My technical research seeks to address this problem by evaluating the ability of vision-language-action models to improve safety and human-machine interaction. My STS research seeks to analyze past automation initiatives to determine key aspects of successful and sustainable automation strategies.
My technical research deals with the problem of automated technologies increasing human-machine conflicts in manufacturing environments. These range from humans struggling to interpret machine output, to machines misinterpreting human actions and causing injuries. As the extent of automation expands, these injuries and breakdowns have the potential to worsen. To address this, my capstone research seeks to determine if robots controlled by vision-language-action (VLA) models can truly improve safety and human-robot collaboration. Such robots are capable of learning from live video, human language, and feedback from their own movements. To do this, we researched several popular VLA models and selected an open-source model known as smolVLA for physical testing and evaluation due to its accessibility. We trained this model to operate a mobile robot and perform approximations of basic industrial tasks including material sorting. Through our training and evaluation of this model, we found that VLA-operated robots may be able to significantly improve the safety of automated systems and integrate human language, but not without extensive resources and time.
My STS research deals with the problem of companies frequently using flawed automation strategies that ultimately do more harm than good. Such strategies have caused factory closures, mass unemployment, and even regional population decline. To do this, my research seeks to uncover the key aspects of successful and sustainable automation. Data from notable case studies of successful and failed automation strategies, worker, manager, and government surveys, and field studies of industrial regions were brought together and analyzed through the framework of Infrastructure Theory (Star, 1999). This analysis was used to uncover key facilitators of successful worker adjustment, employee satisfaction, and minimal economic losses in automation strategies. Through this research, I found that the three most important priorities of any manufacturing automation strategy should be reskilling, effective framing and communication, and thoughtful balancing of stakeholder priorities. I was able to show how automation strategies lacking even one of these aspects became more costly and harmful.
My technical research was able to show that VLA models have some promise in automating industrial tasks and improving safety in industrial environments. However, we were only able to evaluate one lightweight VLA model with limited training and testbenches. In order to determine whether these models can truly benefit human-machine interaction in industry, larger-scale models must be evaluated with more training data and more complex industrial tasks. Significantly, my STS research was able to identify three essential aspects for successful automation strategies and reinforce the importance of each one with real-world examples and evidence in industrial automation. These findings may be able to help companies revise their automation strategies for greater long-term success. Despite this, my research fails to address new complexities associated with advanced AI robotics systems. These technologies will become significant in future automation, and their effects must be thoroughly explored to avoid repeating past failures.
Thank you to my Capstone teammates Jeanu Joo, Andrew Hwang, and Philip Fitz-Gerald for all the effort, skill, and insight they provided to our VLA research as well as in our construction, training, and evaluation of our mobile robot. I would also like to thank Professor Keith Williams for advising us throughout our capstone project and for helping us push through the roadblocks we encountered in our design process. Finally, I would like to thank Professor Caitlin Wylie for teaching me how to research, analyze, and write about sociotechnical issues.
