The wisdom of the crowd: reliable deep reinforcement learning through ensembles of Q-functions
| dc.contributor.author | Elliott, Daniel L., author | |
| dc.contributor.author | Anderson, Charles W., advisor | |
| dc.contributor.author | Draper, Bruce, committee member | |
| dc.contributor.author | Kirby, Michael, committee member | |
| dc.contributor.author | Chong, Edwin, committee member | |
| dc.date.accessioned | 2018-09-10T20:05:46Z | |
| dc.date.available | 2018-09-10T20:05:46Z | |
| dc.date.issued | 2018 | |
| dc.description.abstract | Reinforcement learning agents learn by exploring the environment and then exploiting what they have learned. This frees the human trainers from having to know the preferred action or intrinsic value of each encountered state. The cost of this freedom is reinforcement learning can feel too slow and unstable during learning: exhibiting performance like that of a randomly initialized Q-function just a few parameter updates after solving the task. We explore the possibility that ensemble methods can remedy these shortcomings and do so by investigating a novel technique which harnesses the wisdom of the crowds by bagging Q-function approximator estimates. Our results show that this proposed approach improves all tasks and reinforcement learning approaches attempted. We are able to demonstrate that this is a direct result of the increased stability of the action portion of the state-action-value function used by Q-learning to select actions and by policy gradient methods to train the policy. Recently developed methods attempt to solve these RL challenges at the cost of increasing the number of interactions with the environment by several orders of magnitude. On the other hand, the proposed approach has little downside for inclusion: it addresses RL challenges while reducing the number interactions with the environment. | |
| dc.format.medium | born digital | |
| dc.format.medium | doctoral dissertations | |
| dc.identifier | Elliott_colostate_0053A_15075.pdf | |
| dc.identifier.uri | https://hdl.handle.net/10217/191477 | |
| dc.language | English | |
| dc.language.iso | eng | |
| dc.publisher | Colorado State University. Libraries | |
| dc.relation.ispartof | 2000-2019 | |
| dc.rights | Copyright and other restrictions may apply. User is responsible for compliance with all applicable laws. For information about copyright law, please see https://libguides.colostate.edu/copyright. | |
| dc.subject | machine learning | |
| dc.subject | Q-learning | |
| dc.subject | ensemble | |
| dc.subject | reinforcement learning | |
| dc.subject | neural networks | |
| dc.title | The wisdom of the crowd: reliable deep reinforcement learning through ensembles of Q-functions | |
| dc.type | Text | |
| dcterms.rights.dpla | This Item is protected by copyright and/or related rights (https://rightsstatements.org/vocab/InC/1.0/). You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s). | |
| thesis.degree.discipline | Computer Science | |
| thesis.degree.grantor | Colorado State University | |
| thesis.degree.level | Doctoral | |
| thesis.degree.name | Doctor of Philosophy (Ph.D.) |
Files
Original bundle
1 - 1 of 1
Loading...
- Name:
- Elliott_colostate_0053A_15075.pdf
- Size:
- 29.19 MB
- Format:
- Adobe Portable Document Format
- Description: