Subnetwork ensembles
| dc.contributor.author | Whitaker, Timothy J., author | |
| dc.contributor.author | Whitley, Darrell, advisor | |
| dc.contributor.author | Anderson, Charles, committee member | |
| dc.contributor.author | Krishnaswamy, Nikhil, committee member | |
| dc.contributor.author | Kirby, Michael, committee member | |
| dc.date.accessioned | 2024-01-01T11:25:23Z | |
| dc.date.available | 2024-01-01T11:25:23Z | |
| dc.date.issued | 2023 | |
| dc.description.abstract | Neural network ensembles have been effectively used to improve generalization by combining the predictions of multiple independently trained models. However, the growing scale and complexity of deep neural networks have led to these methods becoming prohibitively expensive and time consuming to implement. Low-cost ensemble methods have become increasingly important as they can alleviate the need to train multiple models from scratch while retaining the generalization benefits that traditional ensemble learning methods afford. This dissertation introduces and formalizes a low-cost framework for constructing Subnetwork Ensembles, where a collection of child networks are formed by sampling, perturbing, and optimizing subnetworks from a trained parent model. We explore several distinct methodologies for generating child networks and we evaluate their efficacy through a variety of ablation studies and established benchmarks. Our findings reveal that this approach can greatly improve training efficiency, parametric utilization, and generalization performance while minimizing computational cost. Subnetwork Ensembles offer a compelling framework for exploring how we can build better systems by leveraging the unrealized potential of deep neural networks. | |
| dc.format.medium | born digital | |
| dc.format.medium | doctoral dissertations | |
| dc.identifier | Whitaker_colostate_0053A_18128.pdf | |
| dc.identifier.uri | https://hdl.handle.net/10217/237458 | |
| dc.language | English | |
| dc.language.iso | eng | |
| dc.publisher | Colorado State University. Libraries | |
| dc.relation.ispartof | 2020- | |
| dc.rights | Copyright and other restrictions may apply. User is responsible for compliance with all applicable laws. For information about copyright law, please see https://libguides.colostate.edu/copyright. | |
| dc.subject | machine learning | |
| dc.subject | ensemble learning | |
| dc.subject | neural networks | |
| dc.title | Subnetwork ensembles | |
| dc.type | Text | |
| dcterms.rights.dpla | This Item is protected by copyright and/or related rights (https://rightsstatements.org/vocab/InC/1.0/). You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s). | |
| thesis.degree.discipline | Computer Science | |
| thesis.degree.grantor | Colorado State University | |
| thesis.degree.level | Doctoral | |
| thesis.degree.name | Doctor of Philosophy (Ph.D.) |
Files
Original bundle
1 - 1 of 1
Loading...
- Name:
- Whitaker_colostate_0053A_18128.pdf
- Size:
- 3.92 MB
- Format:
- Adobe Portable Document Format