SlapBot: The Automated Slapjack Robot; About-Face: The Two Sides of Facial Recognition

Introduction:
The connection between the following two works is that they are both about image recognition technology. For my Capstone project, my team and I used image recognition to identify different types of playing cards, while for STS I did a comprehensive review of facial recognition technology and its biases. Both projects explore the shortcomings of the technology: our Capstone project reviews its potential for misidentification and ineffectiveness through the lens of application and hands-on work, while my STS project studies it from a critical point of view in terms of implications and the way its ineffectiveness impacts the technology in society. For our Capstone project, we designed a robot to play the card game Slapjack; we chose this topic because everyone in our group enjoyed playing the game. As for my STS paper, my motivation for pursuing my topic stemmed from the fact that facial recognition is related to image recognition, as well as having had some previous experience with the topic in prior STS classes.

Capstone Project Summary:
For my technical paper, my Capstone team and I give a review of our work on the SlapBot, our automated Slapjack robot. Slapjack is a card game in which players place cards face-up into a single pile, and compete to be the first to slap a Jack card when it appears on the pile. The key to winning in Slapjack is quick reflexes, but for those who may be unable to react as quickly as others—whether it is due to old age or physical impairments—the competitive nature can be discouraging. The goal of SlapBot was to extend the ability to play Slapjack to more people by creating an automated training partner. This was to allow individuals to practice and improve their reflexes before playing with a group. It used computer vision for image recognition, a robotic arm for the reaction, an STM microcontroller to control the motorized arm through a servo motor, and a Raspberry Pi with a camera module, to identify the Jack card and prompt the arm through the STM32 microcontroller to slap the pile. Through OpenCV and the Raspberry Pi camera, the Raspberry Pi identifies the appearance of a Jack card and sets a GPIO pin high. Overall, the project was a success, with the project having a reasonable success rate (over 75%). The rate of success worked well for the game, since it gives players opportunities to win while also challenging them.

STS Research Paper Study:
What if I told you…you were being watched right now? What once seemed like a sci-fi surveillance fantasy is now a reality, thanks to the development of facial recognition technology (FRT). From identifying suspects to unlocking cell phones, using cameras for facial recognition has found its way into the mainstream; however, it’s not all as utopian as its made out to be. In my STS paper, I explore the applications and repercussions of FRT within the scope of its functional biases, and how its implementation can perpetuate existing biases in society. Case studies include the origin of FRT and its creator’s vision, its use by police enforcement in public for security, a man’s wrongful arrest based on an FRT result, and experiments researching the biases present in the technology, most namely racial and gender biases. This study makes use of the SCOT framework to show how FRT has developed alongside society, and explore the ethics behind its current uses. Outcomes of this research include the consequences of the biased results of FRT, the implications behind the technology’s privacy and data collection policies, and potential solutions to these shortcomings, which include diversifying data sets, sometimes with artificially-generated face data. This research is significant because it gives a critical analysis of a newer technology, and presents ways to improve the ethics involved in its development.

Concluding Reflection:
Working on these two projects together helped to give me a perspective on how the technology works, and why it fails the way that it does when used in public. While we were working on the SlapBot, the image recognition program we used did not have the most difficulty detecting card patterns, but rather the cards themselves. For example, when the environment around the card was not adequate for the algorithm, like if the background was not dark, the reader would return an unknown value. Despite that, the algorithm was able to discern between different kinds of cards when they were properly placed, since it had all the data it needed to do so. Thus, the same could be said for facial recognition algorithms, as they are fundamentally the same: to be able to identify a wide variety of faces, the algorithm requires the necessary data set to train on. Additionally, the lack of proper camera environment could be a reason why public FRT’s fail: if the image it produces is not clear enough, it cannot make an accurate identification, leading to either false positives or negatives. To conclude, these projects could have been done separately, but doing them together helped me gain a holistic understanding of image recognition as a technology.

School of Engineering and Applied Science
Bachelor of Science in Computer Engineering
Technical Advisor: Todd DeLong
STS Advisor: Bryn Seabrook
Technical Team Members: Alex Beck, Aimee Kang, Michael Sekyi