Hardening and Scaling Learned Systems in the Cloud 6 views

Modern data-intensive and AI/ML workloads, particularly those driven by large language models (LLMs), exhibit highly dynamic and bursty behaviors that fundamentally challenge the design assumptions of existing cloud infrastructures. Current systems are largely optimized for static and average-case workloads, resulting in significant inefficiencies in robustness and scalability when deployed in production. This mismatch between workload characteristics and system design introduces critical challenges across both data management and inference serving. In this dissertation, we address these challenges by developing learned systems that are robust under worst-case inputs and efficient under dynamic workloads. First, we study the worst-case complexity behaviors of dynamic learned index structures (LIS), showing hat structured input patterns can trigger substantial memory and CPU overhead. Second, we design λScale, a serverless inference platform that enables “execute-while-load” by overlapping model transfer with distributed execution, achieving fast and cost-efficient scaling under bursty workloads. Together, these contributions demonstrate that addressing both worst-case robustness and dynamic scalability is essential for building next-generation learned systems. This works shows that carefully designing learned components and system architectures to handle real-world workload behaviors can significantly improve the efficiency, scalability, and reliability of learned systems in real-world cloud environments.

PHD (Doctor of Philosophy)

English

All rights reserved by the author (no additional license for public reuse)

2026-04-23