Characterizing and Mitigating Overheads of Distributed Application

Today's applications, such as social networks, data analytics, and retrieval search, rely on collaboration between devices at the user’s end (edge devices) and powerful cloud systems. However, optimizing these systems remains a challenge. First, these cloud-scale systems comprise multiple large-scale distributed services whose communication characteristics are not yet well understood. Secondly, most datacenter traffic involves interactions with storage services, where remote data access includes not only storage device access and data processing, but also RPC stack and queuing latencies. Third, these applications must serve end users on edge devices such as mobile phones. The network latency between edge and cloud and the processing latency within the cloud could be unpredictable. Although local processing does not suffer from this variation, it suffers from limited computing resources such as memory capacity.

This work addresses these three critical challenges in edge-cloud systems. First, we study the overhead of communication within real-world cloud systems to understand and pinpoint potential performance bottlenecks. Secondly, we mitigate the latency of the RPC stack in remote data access in the datacenter by storing persistent data on network devices, effectively moving the latency of the server out of the critical path. Finally, we address the problem of limited resource on edge devices for running data-intensive applications by introducing a memory efficient Retrieval Augmented Generation (RAG) system that allows retrieval search of large database locally on resource-constrained edge devices.