Toward Improving the Performance of Scientific and Desktop/Edge Applications

The key protocols, TCP and IP, underlying the Internet were invented and introduced into ARPAnet, a precursor to the Internet, in the 1970s. The very success of these protocols has constrained the introduction of new high-throughput, low-latency and/or scheduled-delivery network services. While many green-field network designs have proven to be better suited for these types of high-performance services, these designs have been difficult to deploy incrementally into the Internet. Therefore, in this study, we designed and evaluated new networking services, taking into account deployment constraints, so that these services can be introduced incrementally into different regions of the Internet for improved application performance. This is an evolutionary approach to enabling services on deployed networks to improve application performance rather than a revolutionary "design-a-new-network" approach.
Given the large number of network technologies and even larger number of deployed networks, we selected networks within the following three categories: (i) datacenter networks, (ii) Wide-Area Networks (WANs), and (iii) Local-Area Networks (LANs). For each network type, we defined problems that address specific application needs, and proposed and evaluated our evolutionary solutions. For the datacenter networks, we tackled the problem of measuring congestion in InfiniBand production clusters, where congestion is known to reduce the performance of parallelized applications. For WANs, we proposed new network services to support high throughput for large data transfers, and scheduled delivery for delay-sensitive transfers. Finally, for LANs, we focused on the performance of Virtual Desktop (VD) applications. New methodologies are needed to evaluate advances in VD technologies, the results of which would allow for better engineering of VD deployments for improved application performance.
In datacenter networks, low-latency communications are required for scientific, highly parallelized applications. Furthermore, predictable execution times are essential for workflow management. Since HPC clusters deployed by the scientific community use InfiniBand networks, our study targets these networks. Network congestion has been identified as one of the main reasons for performance variability of highly parallelized applications. Characterizing network congestion events will help network administrators identify bottlenecks, and accordingly deploy congestion-control solutions. We developed a methodology for measuring congestion and executed this methodology in a production, highly utilized, InfiniBand cluster called Yellowstone, in which congestion control is currently disabled due to a lack of proven techniques for countering congestion.
Methods for achieving high throughput across WANs are necessary for decreasing transfer times of large datasets. WAN provider links are often operated at low utilization levels, which leaves large unused capacity (headroom). Leveraging this observation, we propose using Software Defined Networking (SDN) controllers to support novel Static Headroom (SH) and Dynamic Headroom (DH) services. These services allow customers to fill the headroom and achieve high throughput without adversely affecting the provider's ability to meet its Best-Effort (BE) service-level agreements. Our solution calls for the use of lower-priority service for large data transfers, and is designed to operate in currently deployed networks with minimal changes.
To allow scheduled delivery of large datasets across WANs, we developed Calibers: Calendar and Large- scale Bandwidth Event-driven Simulations. Calibers leverages SDN-based network architectures and flow pacing algorithms. It uses techniques to intelligently and dynamically shape flows to maximize the number of flows that meet their deadlines while simultaneously improving network resource utilization.
LAN connections are used to offer users VDs from edge-cloud computers to user-owned I/O devices. Zero clients are custom, hardware units with no CPUs, which are designed to enable high-performance delivery of VDs to user-owned I/O devices. We measured the performance of VDs accessed through zero clients to study the feasibility of using this solution to provide desktop-PC experience for disadvantaged communities as a part of a smart-city service. Current deployments of such services are constrained by the inability to run monitoring software packages at the zero clients to assess performance. Therefore, we introduced a new metric and methodology to measure VD performance based on analyzing network traffic, and conducted objective and subjective studies to explore the feasibility of such a solution to provide users a desktop-PC experience from edge clouds.