Repository logo
 

Pruning visual transformers to increase model compression and decrease inference time

Simple item page
dc.contributor.authorYost, James E., author
dc.contributor.authorWhitley, Darrell, advisor
dc.contributor.authorGhosh, Sudipto, committee member
dc.contributor.authorBetten, Anton, committee member
dc.date.accessioned2024-05-27T10:31:53Z
dc.date.available2024-05-27T10:31:53Z
dc.date.issued2024
dc.description.abstractWe investigate the efficacy of pruning a visual transformer during training to reduce inference time while maintaining accuracy. Various training techniques were explored, including epoch-based training, fixed-time training, and training to achieve a specific accuracy threshold. Results indicate that pruning from the inception of training offers significant reductions in inference time without sacrificing model accuracy. Different pruning rates were evaluated, demonstrating a trade-off between training speed and model compression. Slower pruning rates allowed for better convergence to higher accuracy levels and more efficient model recovery. Furthermore, we examine the cost of pruning and the recovery time of pruned models. Overall, the findings suggest that early-stage pruning strategies can effectively produce smaller, more efficient models with comparable or improved performance compared to non-pruned counterparts, offering insights into optimizing model efficiency and resource utilization in AI applications.
dc.format.mediumborn digital
dc.format.mediummasters theses
dc.identifierYost_colostate_0053N_18226.pdf
dc.identifier.urihttps://hdl.handle.net/10217/238378
dc.languageEnglish
dc.language.isoeng
dc.publisherColorado State University. Libraries
dc.relation.ispartof2020-
dc.rightsCopyright and other restrictions may apply. User is responsible for compliance with all applicable laws. For information about copyright law, please see https://libguides.colostate.edu/copyright.
dc.subjectmachine learning
dc.subjecttransformer
dc.subjectpruning
dc.subjectartificial intelligence
dc.titlePruning visual transformers to increase model compression and decrease inference time
dc.typeText
dcterms.rights.dplaThis Item is protected by copyright and/or related rights (https://rightsstatements.org/vocab/InC/1.0/). You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).
thesis.degree.disciplineComputer Science
thesis.degree.grantorColorado State University
thesis.degree.levelMasters
thesis.degree.nameMaster of Science (M.S.)

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Yost_colostate_0053N_18226.pdf
Size:
1.04 MB
Format:
Adobe Portable Document Format
Download

Collections

2020-
Theses and Dissertations