An Automated and Scalable Approach to Hint Generation Using Deep Reinforcement Learning

The rise of technology in recent decades has led to an increased interest in the field of Computer Science particularly from young students. This has caused educators to include more technology-based courses in their curriculum. In some counties across the United States, programming courses are now included in the standard offering in primary, secondary, and high schools. One recurring problem for educators, particularly those teaching programming courses, is the process of guiding students in the right direction (i.e towards a solution to a problem). The faculty to student ratio in most classrooms makes it challenging for educators to constantly be present for each student and personally guide them through problems, thus creating a need for an automatic guidance system, or hint generation framework.

One of the most common methods for solving problems such as hint generation which has a defined space in which there is a discrete set of valid actions, and the solution(s) are known ahead of time is known as Reinforcement Learning. In this work I aim to prove that one block-based programming framework, \textit{Tunescope} developed at the University of Virginia can be modeled as a Reinforcement Learning problem, and therefore prove programming frameworks themselves can be modeled in a way that can be solved programmatically. I continue to solve this problem using a particular type of Reinforcement Learning called Deep Reinforcement Learning (DRL). By solving the problem using DRL, I demonstrate (1) the hint generation problem can be solved programmatically and (2) the solution itself is scalable enough to be applied to other more complex problems.