Generating Synthetic Training Images for Deep Reinforcement Learning of a Mobile Robot
Keywords:
variational autoencoder, deep reinforcement learning, machine learningAbstract
This paper proposes the use of variational autoencoder (VAE) to generate synthetic training images for deep reinforcement learning of a mobile robot. Deep reinforcement learning typically requires millions of interactions with the real world in order to learn good control policies, which is impractical for robotic tasks. Using synthetic images generated by a VAE, one may be able to reduce the number of interactions by running a deep reinforcement learning algorithm off these images, instead of real ones captured by the camera. Our experiment shows, for this particular task, that the VAE can generate synthetic images which are almost non-discernible from those obtained by direct reconstruction of real images.
References
J.-A. M. Assael, N. Wahlstrom, T. B. Sch ¨ on, and M. P. Deisenroth. ¨Data-efficient learning of feedback policies from image pixels using deep dynamical models. arXiv preprint arXiv:1510.02173, 2015.
L. Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Processing Magazine, 29(6):141–142, 2012.
C. Doersch. Tutorial on variational autoencoders. arXiv preprint arXiv:1606.05908, 2016.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672–2680, 2014.
G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313(5786):504–507, 2006.
D. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
D. P. Kingma and M. Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
S. Lange, M. Riedmiller, and A. Voigtlander. Autonomous reinforcement learning on raw visual input data in a real world application. In Neural Networks (IJCNN), The 2012 International Joint Conference on, pages 1–8. IEEE, 2012.
V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013.
V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529–533, 2015.
N. Wahlstrom, T. B. Sch ¨ on, and M. P. Deisenroth. From pixels to ¨ torques: Policy learning with deep dynamical models. arXiv preprint arXiv:1502.02251, 2015.
M. J. Wainwright, M. I. Jordan, et al. Graphical models, exponential families, and variational inference. Foundations and Trends R in Machine Learning, 1(1–2):1–305, 2008.
M. Watter, J. Springenberg, J. Boedecker, and M. Riedmiller. Embed to control: A locally linear latent dynamics model for control from raw images. In Advances in Neural Information Processing Systems, pages 2728–2736, 2015.
F. Zhang, J. Leitner, M. Milford, B. Upcroft, and P. Corke. Towards vision-based deep reinforcement learning for robotic motion control. arXiv preprint arXiv:1511.03791, 2015.
