Satori: A Course Management System; Data Bias Considerations for Artificial Intelligence Systems in Higher Education 521 views

SOCIOTECHNICAL SYNTHESIS As online tools become increasingly prevalent in higher education environments, there is a need for easy-to-use web tools that can adopt the newest technologies in a responsible way. To that end, the technical report details the development and deployment of a new online course tool called Satori for CS classes at the University of Virginia (UVA). The new tool was built in order to allow for more efficient queueing of students in office hours which has been plagued with long wait times; additionally, the tool is built using modern tools that ensure that it can easily be enhanced in the future. The STS research paper is concerned with examining data bias in Artificial Intelligence (AI) systems through the sociotechnical lens of Actor-Network-Theory; an approach that was adapted from a similar examination by STS scholars John Law and Michael Callon. Currently, AI systems are rapidly being adopted into educational contexts without a thorough examination of what the consequences of a biased AI would be in higher education. Additionally, both projects are connected through the choice to examine AI data bias within the context of higher education. As AI systems are inevitably added to Satori, it is imperative that proper consideration is given to how to eliminate data bias within the systems to limit negative consequences. The technical research project is concerned with the replacement of an old online course tool for CS 2150 at UVA for a new one built from the ground up with modern web development tools and techniques. As the shortcomings of the old system were identified, the immediate need for a new tool became apparent. The core functionality of the old system was quickly reproduced within the new system and was immediately available to students with additions added after. Development of the system was done under the supervision of professor Aaron Bloomfield, using the Python based Django web framework. Satori’s success was determined from anectodical feedback from CS 2150 teaching assistants. The feedback has been positive with helpful suggestions for the future of the project. The results and the lessons learned from development have led to the conclusion that although Satori is a significant first step in creating a more helpful online tool for CS classes at UVA, more work must be done to spread the tool to additional courses. The STS research aims to answer the question of: “What considerations should be made to collect unbiased data for AI systems in U.S. universities?”. The paper posits that there is a concerning lack of attention given to the underlying data and processes of AI systems. This was explored through a thorough examination of newspaper and STS journal articles on data bias in AI systems that examine both the underlying causes of the data bias as well as what steps have been taken to address this issue. The primary motivation for the STS research is the trend in AI development where systems are granted autonomy from its creators, alleviating them from the negative consequences of their system’s behavior. To address this, the common issue of data bias within AI systems was examined using Actor-Network-Theory to expose the socio-technical nature of these systems, with a heavy emphasis on responsibility and transparency. The research suggests that in addition to developers taking responsibility for their systems, the European Union’s General Data Protection Regulation should be expanded on and adopted by AI regulators in the future The technical research addresses a current problem facing the CS students at UVA and the STS research looks towards the future of the project to ensure the system remains equitable as it is expanded upon. The STS research heavily emphasizes the sociotechnical nature of AI systems, ensuring that responsibility for eliminating bias and dealing with consequences is placed in the hands of the developers.

BS (Bachelor of Science)

Actor-Network-Theory; Artificial Intelligence; AI; Data Bias; Computer Science

School of Engineering and Applied Science Bachelor of Science in Computer Science Technical Advisor: Aaron Bloomfield STS Advisor: Catherine Baritaud Technical Team Members: Ramya Bhaskara

English

All rights reserved (no additional license for public reuse)

2022-05-06