A Data-Driven Recommender for Gamified Classroom Designs; The Underlying Reason for Mixed Results in Gamification Studies

Although students today have access to all of humanity’s knowledge at their fingertips, educators still struggle with motivating them to learn. When the alternative is memorizing irrelevant information in preparation for soul-crushing exams, it is no wonder that pupils prefer to spend their time playing video games or sports. In light of these challenges, gamification stands as an attractive alternative to traditional lecture-based teaching methods. However, companies attempting to capitalize on its novelty have flooded the market with unproven gamified products claiming to benefit student performance, leading teachers and researchers to fall prey to hype and misinformation. My senior thesis seeks to understand why scholars and instructors have trouble successfully implementing gamification, and how we can more objectively understand the effects of specific game elements. Specifically, my research paper examines the shortcomings of current research methods, while my technical report describes how to design gamified learning environments by drawing on past experimental results.

For my capstone project, I designed and created a web application to help teachers learn about gamification and how to adapt it to their classrooms. I constructed the application using the Django web framework, Bootstrap frontend library, and a local SQLite database. In essence, my website consists of three parts: informational entries, a recommender algorithm, and a data management tool. To start, users can read about the basic game design elements and student learning outcomes. Each entry links to pages summarizing related research articles and providing publication information, such as title, author, and year, to help teachers read more on their own. Additionally, under every article description, I included a detailed list of the experimental results to paint the best picture of the study. Next, users can proceed to the “scenario designer” and input their classroom needs—namely desired outcomes, academic subject, and student age. The website’s recommendation algorithm will then suggest a set of elements to best achieve the user’s goals, drawing from the study results stored in the database. Lastly, my website features data entry forms, which allow users to improve the accuracy of the recommender by adding new resources and results to the website’s database. I also created a page for exporting all stored data to JSON and importing other people’s databases to ensure educators can build on each others’ contributions. My hope is that my website can serve as an accessible resource for educators interested in gamification, pointing them in the correct direction before they start sifting through dense academic papers.

Meanwhile, for my STS research paper, I conducted a critical literature review of authors’ experimental design methods to determine why studies on gamification produce inconclusive or conflicting results. I utilized the Technological, Pedagogical, and Content Knowledge (TPACK) framework to interpret my findings from an educator’s perspective. In short, TPACK states that instructors must understand how new pedagogical techniques (such as gamification) interact with existing learning technologies and subject areas before they can successfully integrate those techniques into their classrooms. In my review, first I discovered that researchers tend to test multiple elements together instead of separately in controlled experiments. This lack of scientific rigor prevents authors from understanding how elements individually affect student behavior. Moreover, I identified three learning outcomes dominant in literature—grades, engagement, and motivation (collectively GEM). I argued that scholars’ excessive focus on improving GEM metrics drives them to select sources of extrinsic motivation. Far from increasing student performance, such rewards often cause students to lose interest in learning, appearing as just another goal to grind towards on top of good grades. Worse yet, this overreliance on GEM encourages researchers to follow trends rather than pursue objective knowledge, while dissuading study into under-tested elements or outcomes. Afterward, I discussed how researchers fail to include teacher and student input while designing their experiments. As a result, the gamified system may not be well-suited to the particular classrooms’ needs, preventing instructors from following TPACK. Ultimately, gamification should not be treated as a substitute for proper scientific rigor or magic bullet for immediate improving student achievement. I also believe that future studies should involve teachers and students more actively.

Overall, working on these two projects together has provided me with multiple vital perspectives on gamification—those of students, instructors, researchers, and engineers. In my background research, I uncovered students’ frustrations with the current educational system and their desire for a more engaging experience. Then, through TPACK, I understood how much ease of integration matters to teachers, who are constantly managing multiple skills and areas of knowledge. Furthermore, in my analysis, I explored the difficulties of navigating through academia and how filtering fact from fiction is not always straightforward. Finally, in creating my technical project, I needed to make design decisions to ensure my website closely modeled study results while remaining usable to those without high technological literacy. Understanding all these perspectives at once enabled me to narrow my direction of research and website requirements and create a final product that can benefit a wider audience. Personally, I have long been interested in gamification thanks to my past experiences tutoring younger students and teaching at summer camps. Therefore, I am grateful that working on my thesis has also given me a fulfilling opportunity to investigate a topic I am invested in.

School of Engineering and Applied Science

Bachelor of Science in Computer Science

Technical Advisor: Mark Sherriff

STS Advisors: Rider Foley, MC Forelle