Abstract
It is an undeniable fact that artificial intelligence is among the most significant technological advancements of our generation. It has had and continues to have a profound impact on how both industries and individuals conduct their work and daily lives. It is also undeniable, however, that AI is a very controversial technology, with many voicing concerns about its ethical use in education or how it affects its users mentally. My capstone project somewhat explores the former concern, since I worked on improving the AI-Smart Classroom Initiative (ASCI) system, which is a set of tools designed to promote course engagement and enhance office hours. Though there were also many tools that did not rely on AI, most notably, the ASCI system employed a course chatbot that students could interface with while waiting for an instructor. In addition to working on a system that allows for the ethical use of AI in education, I was also motivated to pursue this project due to my own experiences with office hours in certain courses. My STS research examined the latter concern, since I dove into how the design of chatbots affects their interactions with users and how those interactions affect the mental health of users. I was interested in this topic because, while there have been some articles about particularly harmful AI-human interactions, there did not seem to be any concrete correlations found, despite the effects of AI-human interactions being an increasingly important topic as AI continues to develop. Together, my capstone project and STS research both focus on chatbots, their implementation, use, and effects.
I have spent entire office hours working by myself, which is counterintuitive to attending office hours in the first place. I had no choice in the matter, however, because the reality is that, in some courses, there is a much greater demand for office hour assistance than there is a supply of instructors and TAs to provide it. The ASCI system remedies this issue with a number of different tools, such as an office hours queue to provide structure, student grouping to increase efficiency, and a retrieval-augmented generation (RAG) chatbot that is given course content. Beyond office hours, the ASCI system also improves student engagement by allowing instructors to monitor student progress and discern any individuals who may need additional help, as well as having a “Quest” section that allows for the gamification of extracurricular activities, like attending office hours or a guest lecture. For my capstone project, my team and I built on this system to provide more automation and convenience to the chatbot creation process and fully fleshed out the student monitoring and “Quest” functionalities.
Overall, we were able to implement various additions to the ASCI system, such as the ability to upload individual documents to the chatbot rather than one ZIP file as well as the ability to scrape course webpages to create a TXT file with the relevant content, more comprehensive options to monitor student progress, including a new Discord activity monitor, and changes to how the chatbot would format its responses to make it more in-line with more popular interfaces. After beta testing, we also made general improvements to the UI of the system, like allowing users to collapse introductory information on certain pages to allow for more focus on the main function of the page, since testers were generally satisfied with the functionality but thought the design could be worked on. The stakeholder review provided more insight into areas of improvement like automating the scraper to also upload the TXT file or connecting the queue with the “Quest” system to automatically update office hour quest progress. Upon the second meeting, the stakeholders were also generally satisfied with the functionality as well, however.
For my STS research, I sought to answer the question, “How do the design choices in chatbots shape the ways users experience companionship and support?” I thought that this was an important issue to consider due to the current gaps in knowledge relating to chatbot-human interactions coupled with the increase in chatbot use and chatbot-related incidents. My approach to the question involved examining chatbot design and finding the main design choice that guided chatbot design was anthropomorphism. There were both anthropomorphic chatbots and machine-like chatbots, and so I reviewed the available literature to discern and discuss the pros and cons of each before suggesting what might be the most mindful way to design chatbots moving forward. Throughout my analysis, I also employ the STS concepts of the social construction of technology (SCOT) and latent and manifest functions and dysfunctions to better understand how the design choices were made and their effects.
My findings indicated that, though anthropomorphic chatbots are more effective at forming personal connections with users, this also results in the latent dysfunction of anthropomorphic chatbots being more effective at misleading users or causing them to form unhealthy attachments to them. Machine-like chatbots, while less effective at providing companionship, were still able to provide mental health support and had the latent function of being a non-judgmental safe space for many users, since they did not feel the pressure of conversing with a human. Machine-like chatbots are less popular than anthropomorphic chatbots for obvious reasons; however, I believe that they still provide a niche that is fundamentally impossible for their more “human” counterparts to fulfil and should, thus, not be abandoned as chatbot technology develops. For direction moving forward, I suggest how the use of chatbots could be improved by combining some of the features of both. The stronger connections that anthropomorphic chatbots can form is a double-edged sword, but it could potentially be tempered by including the machine-like chatbots’ clear statements affirming their status as AI chatbots. Regardless, it is worth noting that neither chatbot is a replacement for professional help and likely will never be. As they continue to be used more and more frequently, however, it remains important to study their impact and continue refining them to be the most effective and least harmful as they can be.
