MAGELLAN: enabling effective search over voluminous, high-dimensional scientific datasets

Abstract

As high-dimensional, voluminous datasets continue to become available, they present opportunities for users to perform richer explorations that lead to insights. Most explorations are however limited by the query semantics enforced by the underlying storage system. This precludes identification of connections that exists within and across datasets. This study describes, Magellan, a system that is designed for richer, iterative explorations that allow users to explore connections within and across datasets. Our methodology combines aspects of ontologies and metadata to support analysis that are domain informed and statistically richer. Our performance benchmarks demonstrate the suitability of our methodology to inform explorations interactively and at scale.