Designing Time Series Data Storage Systems that Balance Performance, Usability, and Multi-Tenancy

The number of Internet of Things, or IoT, devices is growing rapidly (a 214% increase from 2015 to 2020). Many expect the trend to continue, with some forecasts predicting 140% growth through 2025. An area which stands to benefit from this expansion is the deployment of time-series data-collecting devices, however, current methods of storing and interacting with time series data are typically limited in one of two ways. Some systems are user-friendly at the cost of query performance or maximum ingest rate. Others have great performance but a more difficult user experience. To get around these issues, we propose a new method for designing time series data management systems. Our method takes isolated features that users desire, like flexible data ingestion and automated database configuration, and offers guidance for how to combine them with a simple and scalable back-end architecture. Using the new method, we built the Smart Infrastructure Foundation (SIF), a system optimized for the collection and storage of sensor data from University buildings, which improves upon existing solutions by offering a greatly simplified user experience without sacrificing generality, scalability, or efficiency.

Funding for this project was provided by the University of Virginia's Strategic Investment Fund.