Abstract
Mental health disorders affect over one billion people globally, yet support systems remain largely reactive—relying on scheduled appointments or active self-report to detect distress. Passive behavioral sensing from smartphones and wearables offers a path toward proactive support, but translating noisy sensor streams into actionable clinical insight remains an open challenge. This dissertation investigates how wearable sensing, large language models (LLMs), and agentic AI can be progressively combined to build context-aware, proactive systems for mental health.
Study 1 (SIMS) establishes that wearable sensors can detect social contexts relevant to mental health. In a controlled experiment with 46 socially anxious individuals, we demonstrate that passively sensed biobehavioral signals can detect whether a person is in a social situation (78.0% balanced accuracy), the group size (61.8%), and the temporal phase of interaction (78.3%). We find that voice features capture evaluative social threat that physiological sensors alone cannot, highlighting the importance of multimodal sensing.
Study 2 (CALLM) moves from detection to interpretation. Using phone sensing data from 407 cancer survivors in the BUCS longitudinal study, we introduce CALLM, a retrieval-augmented LLM framework that infers affect and intervention readiness from ultra-brief digital diary entries (median 4 words). CALLM achieves 73.7% balanced accuracy for emotion regulation desire, 73.3% for negative affect, 73.0% for positive affect, and 60.1% for intervention availability, significantly outperforming traditional NLP and zero-shot LLM baselines—but remains reactive.
Study 3 (PULSE) asks whether giving the LLM a full harness—raw sensor data, purpose-built tools, and investigative autonomy—enables it to go further. Building on the same BUCS dataset with six passive smartphone sensing modalities, we introduce PULSE, a framework in which LLM agents autonomously investigate behavioral data through eight purpose-built sensing tools. A 2 * 2 factorial evaluation on 50 randomly selected cancer survivors (4,112 predictions per system) reveals that agentic reasoning yields consistent gains over structured pipelines: autonomous agents outperform both supervised-learning and LLM baselines (p < 0.0001), achieving 72.5% mean balanced accuracy. Emotion regulation desire benefits substantially from diary text (74.3% vs.\ 65.1%), while intervention availability is predicted strongly from passive sensing alone (71.3%)—the most practically important target for just-in-time adaptive interventions.
Together, these studies demonstrate a progression from sensing to understanding to agentic assistance, contributing new methods and empirical evidence toward AI that can observe, reason about, and proactively respond to human behavior for mental health support.
