Optimal chaos control through reinforcement learning

Abstract

We present an approach for the control of chaotic systems based on reinforcement learning. Reinforcement learning algorithms provide a solution to the optimal control of systems in situations where system dynamics and analytic information about the desired goal state are not available. We formulate the chaos control problem as a reinforcement learning problem to obtain a general purpose chaos controller which acts globally, allows control in non-stationary and noisy environments, and can be used in parametric or impulsive control applications. To reduce the computational complexity of the reinforcement learning problem, vector quantization techniques are applied and the control policy is approximated in the reduced space. To find minimal sufficient codebooks, we suggest a fixed-point sensitive growing chaos controller which combines a modification of Fritzke's Growing Neural-Gas algorithm with reinforcement learning. We demonstrate the algorithm in a variety of applications including low- and high-dimensional discrete and chaotic systems, logistic coupled map lattices and a multistable rotor.