Enabling programmatic interfaces for explorations over voluminous spatiotemporal data collections

Abstract

This thesis focuses on enabling programmatic interfaces to perform exploratory analyses over voluminous data collections. The data we consider can be encoded in diverse formats and managed using diverse data storage frameworks. Our framework, Scrybe, manages the competing pulls of expressive computations and the need to conserve resource utilization in shared clusters. The framework includes support for differentiated quality of services allowing preferentially higher resource utilization for certain users. We have validated our methodology with voluminous data collections housed in relational, NoSQL/document, and hybrid storage systems. Our benchmarks demonstrate the effectiveness of our methodology across evaluation metrics such as latencies, throughputs, preservation of resource thresholds, and differentiated services. These quantitative measures of performance are complemented using qualitative metrics that profile user interactions with the framework.