Late residual neural networks: an approach to combat the dead ReLU problem
| dc.contributor.author | Ernst, Matthew Frederick, author | |
| dc.contributor.author | Whitley, Darrell, advisor | |
| dc.contributor.author | Anderson, Chuck, committee member | |
| dc.contributor.author | Buchanan, Norm, committee member | |
| dc.date.accessioned | 2022-05-30T10:21:03Z | |
| dc.date.available | 2022-05-30T10:21:03Z | |
| dc.date.issued | 2022 | |
| dc.description.abstract | The rectified linear unit (ReLU) activation function has been a staple tool in deep learning to increase the performance of deep neural network architectures. However, the ReLU activation function has trade-offs with its performance, specifically the dead ReLU problem caused by vanishing gradients. In this thesis, we introduce "late residual connections" a type of residual neural network with connections from each hidden layer connected directly to the output layer of a network. These residual connections improve convergence for neural networks by allowing more gradient flow to the hidden layers of a network. | |
| dc.format.medium | born digital | |
| dc.format.medium | masters theses | |
| dc.identifier | Ernst_colostate_0053N_17004.pdf | |
| dc.identifier.uri | https://hdl.handle.net/10217/235155 | |
| dc.language | English | |
| dc.language.iso | eng | |
| dc.publisher | Colorado State University. Libraries | |
| dc.relation.ispartof | 2020- | |
| dc.rights | Copyright and other restrictions may apply. User is responsible for compliance with all applicable laws. For information about copyright law, please see https://libguides.colostate.edu/copyright. | |
| dc.subject | residual connections | |
| dc.subject | dead ReLU | |
| dc.title | Late residual neural networks: an approach to combat the dead ReLU problem | |
| dc.type | Text | |
| dcterms.rights.dpla | This Item is protected by copyright and/or related rights (https://rightsstatements.org/vocab/InC/1.0/). You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s). | |
| thesis.degree.discipline | Computer Science | |
| thesis.degree.grantor | Colorado State University | |
| thesis.degree.level | Masters | |
| thesis.degree.name | Master of Science (M.S.) |
Files
Original bundle
1 - 1 of 1
Loading...
- Name:
- Ernst_colostate_0053N_17004.pdf
- Size:
- 2.76 MB
- Format:
- Adobe Portable Document Format