Online Archive of University of Virginia Scholarship
Hardening and Scaling Learned Systems in the Cloud6 views
Author
Yang, Rui, Computer Science - School of Engineering and Applied Science, University of Virginia0000-0002-6522-6305
Advisors
Cheng, Yue, DS-Faculty Affairs, University of Virginia
Abstract
Modern data-intensive and AI/ML workloads, particularly those driven by large language models (LLMs), exhibit highly dynamic and bursty behaviors that fundamentally challenge the design assumptions of existing cloud infrastructures. Current systems are largely optimized for static and average-case workloads, resulting in significant inefficiencies in robustness and scalability when deployed in production. This mismatch between workload characteristics and system design introduces critical challenges across both data management and inference serving.
In this dissertation, we address these challenges by developing learned systems that are robust under worst-case inputs and efficient under dynamic workloads. First, we study the worst-case complexity behaviors of dynamic learned index structures (LIS), showing hat structured input patterns can trigger substantial memory and CPU overhead. Second, we design λScale, a serverless inference platform that enables “execute-while-load” by overlapping model transfer with distributed execution, achieving fast and cost-efficient scaling under bursty workloads.
Together, these contributions demonstrate that addressing both worst-case robustness and dynamic scalability is essential for building next-generation learned systems. This works shows that carefully designing learned components and system architectures to handle real-world workload behaviors can significantly improve the efficiency, scalability, and reliability of learned systems in real-world cloud environments.
Degree
PHD (Doctor of Philosophy)
Language
English
Rights
All rights reserved by the author (no additional license for public reuse)
Yang, Rui. Hardening and Scaling Learned Systems in the Cloud. University of Virginia, Computer Science - School of Engineering and Applied Science, PHD (Doctor of Philosophy), 2026-04-23, https://doi.org/10.18130/e0zv-ew31.