REPS by Peters et al., 2010 is an approach to tackle the optimisation bias and information loss often associated with continuous control Reinforcement Learning tasks by introducing an upper bound by which both the policy and value functions are allowed to change. Extending the approach by van Hoof et al., 2017, this research project by Knegt, Kuric, Müller-Eberstein and Scheffers aims to extend REPS to deep neural network models which perform tasks within OpenAI Gym.
This project is written in Python3 and requires the dependencies listed in the requirements.txt file. These include:
Please follow each respective installation guide if necessary.
Scripts to run gym tasks for Pendulum-v0, Swimmer-v2, HalfCheetah-v2 as well as Reacher-v2, Hopper-v2 you can use the apprpriate scripts in the run/ directory.
$ python3 run/pendulum.py
The models of each iteration are saved as .pth files and the evaluation results should be rendered every two iterations.
Rendered results of the best performing model configurations for Pendulum-v0, Swimmer-v2 and HalfCheetah-v2 can be found on YouTube:
Further plots and result numbers can be found within the results/ directory.
Peters, J., Mulling, K., and Altün, Y. Relative entropy policy search. In Fox, M. and Poole, D. (eds.), Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2010), pp. 1607–1612. AAAI Press, 2010.
van Hoof, H., Neumann, G., and Peters, J. Non-parametric policy search with limited information loss. Journal of Machine Learning Research, 18(73):1–46, 2017.
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. Openai G ym. CoRR, abs/1606.01540, 2016.