A framework for resource efficient profiling of spatial model performance
| dc.contributor.author | Carlson, Caleb, author | |
| dc.contributor.author | Pallickara, Shrideep, advisor | |
| dc.contributor.author | Pallickara, Sangmi Lee, advisor | |
| dc.contributor.author | Adams, Henry, committee member | |
| dc.date.accessioned | 2022-08-29T10:15:48Z | |
| dc.date.available | 2022-08-29T10:15:48Z | |
| dc.date.issued | 2022 | |
| dc.description.abstract | We design models to understand phenomena, make predictions, and/or inform decision-making. This study targets models that encapsulate spatially evolving phenomena. Given a model M, our objective is to identify how well the model predicts across all geospatial extents. A modeler may expect these validations to occur at varying spatial resolutions (e.g., states, counties, towns, census tracts). Assessing a model with all available ground-truth data is infeasible due to the data volumes involved. We propose a framework to assess the performance of models at scale over diverse spatial data collections. Our methodology ensures orchestration of validation workloads while reducing memory strain, alleviating contention, enabling concurrency, and ensuring high throughput. We introduce the notion of a validation budget that represents an upper-bound on the total number of observations that are used to assess the performance of models across spatial extents. The validation budget attempts to capture the distribution characteristics of observations and is informed by multiple sampling strategies. Our design allows us to decouple the validation from the underlying model-fitting libraries to interoperate with models designed using different libraries and analytical engines; our advanced research prototype currently supports Scikit-learn, PyTorch, and TensorFlow. We have conducted extensive benchmarks that demonstrate the suitability of our methodology. | |
| dc.format.medium | born digital | |
| dc.format.medium | masters theses | |
| dc.identifier | Carlson_colostate_0053N_17242.pdf | |
| dc.identifier.uri | https://hdl.handle.net/10217/235566 | |
| dc.language | English | |
| dc.language.iso | eng | |
| dc.publisher | Colorado State University. Libraries | |
| dc.relation.ispartof | 2020- | |
| dc.rights | Copyright and other restrictions may apply. User is responsible for compliance with all applicable laws. For information about copyright law, please see https://libguides.colostate.edu/copyright. | |
| dc.title | A framework for resource efficient profiling of spatial model performance | |
| dc.type | Text | |
| dcterms.rights.dpla | This Item is protected by copyright and/or related rights (https://rightsstatements.org/vocab/InC/1.0/). You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s). | |
| thesis.degree.discipline | Computer Science | |
| thesis.degree.grantor | Colorado State University | |
| thesis.degree.level | Masters | |
| thesis.degree.name | Master of Science (M.S.) |
Files
Original bundle
1 - 1 of 1
Loading...
- Name:
- Carlson_colostate_0053N_17242.pdf
- Size:
- 2.66 MB
- Format:
- Adobe Portable Document Format