A Chatbot to Identify Sentence Ambiguity in Conversational Text; Navigating the Associated Risks of Emotion AI Across Different Industries

The desire to establish a better understanding between humans and machines has led to the increasing implementation of emotion recognition technologies. Known as Emotion AI, this technology uses biological data and inputs such as facial expressions, eye movements, body language, vocal patterns, heart rate, and more to identify what emotion a person is feeling. A similar subfield that parses textual data to interpret human languages is referred to as Natural Language Processing (NLP). Both my technical capstone project and STS research analyze the problems that these fields experience in the current landscape. The goal for both projects was to provide insight into the impact of these emerging technologies and offer potential solutions.
The problems that appear with Emotion AI and NLP impact its effectiveness and accuracy. A major issue is the lack of regulation during the development and implementation stages which raises concerns about the potential misuse and ethical implications of these technologies. This directly ties into another major concern about data privacy, where unauthorized data collection and usage is only possible because of the absence of clear guidelines and enforcement. Another substantial problem is the potential for errors and inaccuracies to be produced by the algorithms that could be caused by a variety of factors such as the lack of a fully representative dataset. For NLP, these inaccuracies are typically caused by the ambiguity behind interpreting what an emotion or written sentence means. Addressing these problems is important because it could lead to unintended consequences when integrating these functionalities into daily usage. And in the case where people’s health and safety are on the line, errors can provide more harm than help.
My STS research investigated how different social groups are impacted by the associated risk of using products with Emotion AI. I conducted a literature review of all the existing documented implementations of it in different industries, such as corporate, healthcare, and automotive industries. Corporate industries have integrated Emotion AI into their hiring processes to streamline how efficiently they can recruit new job candidates. They have also incorporated it into their workplace surveillance practices to monitor employee performance and productivity. The healthcare industry uses Emotion AI to help healthcare workers make more informed decisions for diagnoses and treatment plans. And finally, the automotive industry uses it to prevent reckless driving and encourage safer driving practices. The identification of these applications enabled me to identify multiple social groups within those industries that were affected in a significant way, and I analyzed the corresponding risk using the Social Construction of Technology (SCOT) framework. The resulting patterns and trends involving the associated risk consist of the potential of being too reliant on those technologies and how power dynamics between social groups enable the user of the products to inflict more risk on those with less authority.
In response to the established problem with NLP, I proposed the concept of a chatbot that is designed to detect textual inputs that are ambiguous with the goal of encouraging people to provide NLP tools with the proper data and context. The design specifications for this proposition begin with the tokenization of larger text into smaller phrases and words. Then it follows with how the tokenized entities need to be tagged by the part of speech it falls under and fed into a pre-trained transformer model to see how accurately the next word in a sentence can be predicted. The combination of that with the Hidden Markov Models (HMM) contributes to higher accuracy. My main suggestion is to include an ambiguity detector in the resulting chatbot. The inputted text is rated on its ambiguity based on the HMM’s entropy of the predicted tag distribution. The combination of all of this allows the chatbot to respond to the user’s textual input with how ambiguous the meaning is. The production of this chatbot will allow product developers working with Emotion AI to account for ambiguous instructions and errors in judgment to make the technology more accurate.