Same data, same features: modern ImageNet-trained convolutional neural networks learn the same thing
| dc.contributor.author | McNeely-White, David G., author | |
| dc.contributor.author | Beveridge, J. Ross, advisor | |
| dc.contributor.author | Anderson, Charles W., committee member | |
| dc.contributor.author | Seger, Carol A., committee member | |
| dc.date.accessioned | 2020-06-22T11:52:48Z | |
| dc.date.available | 2020-06-22T11:52:48Z | |
| dc.date.issued | 2020 | |
| dc.description.abstract | Deep convolutional neural networks (CNNs) are the dominant technology in computer vision today. Much of the recent computer vision literature can be thought of as a competition to find the best architecture for vision within the deep convolutional framework. Despite all the effort invested in developing sophisticated convolutional architectures, however, it's not clear how different from each other the best CNNs really are. This thesis measures the similarity between ten well-known CNNs, in terms of the properties they extract from images. I find that the properties extracted by each of the ten networks are very similar to each other, in the sense that any of their features can be well approximated by an affine transformation of the features of any of the other nine. In particular, there is evidence that each network extracts mostly the same information as each other network, though some do it more robustly. The similarity between each of these CNNs is surprising. Convolutional neural networks learn complex non-linear features of images, and the architectural differences between systems suggest that these non-linear functions should take different forms. Nonetheless, these ten CNNs which were trained on the same data set seem to have learned to extract similar properties from images. In essence, each CNN's training algorithm hill-climbs in a very different parameter space, yet converges on a similar solution. This suggests that for CNNs, the selection of the training set and strategy may be more important than the selection of the convolutional architecture. | |
| dc.format.medium | born digital | |
| dc.format.medium | masters theses | |
| dc.identifier | White_colostate_0053N_15985.pdf | |
| dc.identifier.uri | https://hdl.handle.net/10217/208467 | |
| dc.language | English | |
| dc.language.iso | eng | |
| dc.publisher | Colorado State University. Libraries | |
| dc.relation.ispartof | 2020- | |
| dc.rights | Copyright and other restrictions may apply. User is responsible for compliance with all applicable laws. For information about copyright law, please see https://libguides.colostate.edu/copyright. | |
| dc.subject | convolutional neural networks | |
| dc.subject | feature space | |
| dc.subject | machine learning | |
| dc.subject | feature mapping | |
| dc.subject | computer vision | |
| dc.subject | ImageNet | |
| dc.title | Same data, same features: modern ImageNet-trained convolutional neural networks learn the same thing | |
| dc.type | Text | |
| dcterms.rights.dpla | This Item is protected by copyright and/or related rights (https://rightsstatements.org/vocab/InC/1.0/). You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s). | |
| thesis.degree.discipline | Computer Science | |
| thesis.degree.grantor | Colorado State University | |
| thesis.degree.level | Masters | |
| thesis.degree.name | Master of Science (M.S.) |
Files
Original bundle
1 - 1 of 1
Loading...
- Name:
- White_colostate_0053N_15985.pdf
- Size:
- 1.66 MB
- Format:
- Adobe Portable Document Format