Developing Adaptive Information Systems: Learning and Evolving Through Human Feedback

Modern information systems, such as recommender systems, are typically characterized by their human-centric designs and adaptiveness, where the development of Human-Feedback-driven Learning (HFL) mechanisms is the central focus. However, most existing works treat human feedback as readily available ground-truth data, yet numerous challenges await dedicated resolutions: 1) From a system’s perspective, real-world human feedback, typically collected from ordinary users, often lacks rigorous quality control. This results in feedback data that is overly noisy and requires sophisticated treatments, such as data cleansing and augmentation before it can be utilized to develop robust systems. 2) From users’ perspective, disappointment arises when systems misinterpret their feedback or fail to react to their needs, leading to lower quality future feedback and hindering system improvement and user satisfaction. In this dissertation, we focus on modeling human feedback from both perspectives. On the one hand, we propose novel frameworks for learning from noisy and sparse human feedback. On the other hand, we devise algorithms that efficiently and effectively learn personalized policies, enabling systems to interpret and elicit users’ interests and intentions through their feedback. We evaluate the effectiveness of proposed methods in various scenarios, including crowdsourcing, recommender systems, and nature language generation.