Moving the Dialogue Forward: Virtual Student Conversational Agent Design in Low Data Environments

Tech giants have spent millions of dollars developing what appear to be intelligent conversational agents, such as Siri and Alexa. While advancing portions of the text analytics field, these applications often rely on vast amounts of pre-programmed tasks and rules-based dialogue policies, as well as a large amount of domain expert input to achieve the illusion of language understanding. The illusions created by either well-funded or straight-forward, closed-domain, task-oriented, and typically customer-service-centric conversational agents have led to a societal-level misconception of what we are truly capable of within the Conversational Agent domain. In domains with insufficient data and funding, the hopes of developing complex, diverse-purposed conversational agents are often unlikely to be realized due to the lack of labeled data, resources, and codified processes that differ from customer-service-oriented design needs.

In this dissertation, I detail the process of developing a meta-purpose conversational agent, specifically a pedagogical teachable agent. This development is one of few meta-purpose agents in the literature and the first pedagogical teachable agent in the literature that incorporateds state-of-the-art Natural Language Processing (NLP) techniques such as incorporating generative responses and free-form natural language user inputs. Users engage in a teacher role when interacting with our virtual student, the AI-based classroom teaching system (ACTS), who needs assistance with a STEM-related problem.

I outline discussion for evaluation needs for non-customer-service agents and the importance of anthropomorphic quality development, such as mimicking the fallibility of understanding more representative of a real-world student. I propose a development framework and provide transparent insight into the development process. Finally, I validate the proposed conversational agent with a novel study involving the assessment of my design. The results of this study contribute to pedagogical conversational agent discussions and the development process for meta-purpose and teachable agents.