Abstract
Technical Project
The use of generative AI among college students is on the rise, and there are growing concerns from both students and instructors regarding the best way to manage these tools in the classroom. Particularly, computer science college professors remain uncertain on whether to limit or integrate generative AI into their curriculum. Especially with the intense spike in AI usage in the computer science industry, professors are unsure how to properly prepare their students for these jobs while still increasing their base knowledge of the subject. In my technical project, I explore how generative AI affects student thinking. Specifically, I answer the following research question: How do novice programmers’ reasoning processes and problem-solving approaches differ when programming with AI code generation compared to without it?
For my research, I performed a qualitative analysis of interview data collected during a study conducted by Nicholas Gardella and coauthors titled “Performance, Workload, Emotion, and Self-Efficacy of Novice Programmers Using AI Code Generation” (2024). They did a within-subject study to examine how various metrics changed when students completed coding problems either with or without GitHub Copilot, an artificial intelligence-driven development environment. I analyzed the interview transcripts that weren’t included in the original study to see how the novice programmers’ thought processes changed when coding with and without Copilot. I chose to conduct a reflexive thematic analysis to study the data. During my analysis, I familiarized myself with the data, generated codes to classify each interview, and used the codes to construct themes about the data as a whole.
The results of this paper aren’t ready for publication, but there are preliminary insights I can share. One theme I identified among student responses was code ownership. Students are typically aware and willing to admit when Copilot takes the lead in developing a solution, versus when they do. Interestingly, this isn’t always tied to how much actual code Copilot versus the student types out. This is because participants in this study used the in-line suggestion feature of Copilot, where it actively offers suggestions of how to finish your code based on what you’ve already written. This means a participant might understand exactly how to solve a problem and begin typing a solution, and when Copilot picks up on their approach, it is able to suggest the rest of it. Students are aware that it was their idea and approach, meaning they are conscious of how they are and aren’t relying on AI.
STS Project
Duolingo is a free and accessible language-learning platform where users can choose from 40+ languages to learn vocabulary, grammar, and conversational skills. It has become one of the world’s leading educational applications to date. Despite these accolades, research challenges the application's effectiveness in teaching a language, its questionable use of gamification, and its overall reputation as a company. Especially given that Duolingo’s mission is to develop quality and accessible educational content, there exists tension between their true effectiveness and this goal.
My STS project answers the following question: How have Duolingo’s gamification elements, business model, use of artificial intelligence (AI) systems, and other actors influenced how well the app has fulfilled its mission of making quality language education accessible to everyone? This question reveals the nature of for-profit companies that aim to bridge social or socioeconomic gaps. It’s important to see how these companies’ strategies create value for shareholders while still carrying out their humanitarian mission. During my research, I used Actor-Network Theory to open up the black box of Duolingo, analyze the actors that have shaped it, and show how certain behaviors have been delegated to various social groups. While the CEO, Luis Von Ahn, and the company's choices have been well-intentioned and consistent with their mission, there are areas for improvement in the application's features and how it affects users. This includes their arguable misuse of gamification and lack of acknowledgement of the true effectiveness of their application.
These two projects explore AI’s impact on two distinct educational realms, institutional and commercial. My STS project reveals the extent to which AI and automated systems are able to effectively teach languages, and my technical project explores how the use of these systems affects students’ reasoning processes. I think that research in both of these areas is important in forming the bigger picture of how AI is shaping education. I hope these projects inform future educational technology companies and universities on the best way to integrate AI into their educational methods.
