Reinforcement Learning for Resource Management; Environmental Construction of Big Data Initiatives

The growing power demand of large-scale ICT services has given rise for the need for both technical and social improvement of Big Data systems. The technical advancements of the past two decades continued to optimize current systems but have not shown an ability to decouple the growth in demand of the services from the growth in overall power use. In order to minimize the potential risks of large-scale ICT, designers must also consider the higher order effects of their products and services. Changes in user patterns or global markets as a result of Big Data’s explosion into world industry could have unforeseen environmental affects, such as the affect replacing a previously manual system with a high-electricity, automated one. To identify paths toward a more responsible design of Big Data initiatives, the contributions of users, ICT providers, and researchers must be considered and critiqued regarding how each may contribute to the resulting environmental dangers.
In the technical portion of my research, the potential of Reinforcement Learning (RL) for resource management was tested in comparison to more traditional resource management systems found in data centers. Expanding on prior research studying the efficiency improvement of policy gradient methods, my research implements advantage actor-critic (AAC) models and tests various reward mechanisms. My findings concluded that AAC models trained with a squared reward function show a potential to reduce worst-case runtime by over 200% in a simulated environment. In the STS portion of my research, the various actors involved in the design of ICT systems were critiqued and suggestions were made for how their cooperation may be improved. My research concludes that researchers and providers must work together in order to achieve reasonable estimates of the net environmental impact. In addition, users must be informed of the direct and indirect consequence of their consumption, and only service providers can provide this education. ICT providers must reconcile that despite appearing as an intangible, decentralized service, large-scale ICT systems are physical technologies with high energy usage. The intangibility of the product must be addressed with the intent of education, rather than being taken advantage of in order to pitch ICT initiatives as eco-friendly.
During this research, it has become apparent that there is no lack of motivation for ICT providers to improve their systems technically. Electricity cost directly affects the profitability of a data center, so providers who can afford it will continue to make the systems more efficient. However, the higher order effects of the technologies are often willfully ignored by all the actors except academics. While technical improvements are likely to continue for the foreseeable future, efforts could be futile if the potential higher order affects are never accurately estimated. Ultimately, it is the users who must force this shift in focus. Users must realize each time they visit a digital marketplace, use their smart TV, or scroll through a customized ad feed, they are actively consuming a product which could have a large negative effect to the planet. Only through a demand by the users can providers be forced to work with academics to estimate how large-scale ICT may be affecting the world, and only with these estimates can we know if large, systemic change is needed in addition to continued technical improvement.

School of Engineering and Applied Science
Bachelor of Science in Computer Science
Technical Advisor: Haiying Shen
STS Advisor: Richard Jacques
Technical Team Members: None