This fork difference from OpenAI Baselines

Currently this fork differs from the OpenAI master branch after commit 8c90f67. The baselines/deepq/experiments/atari module has enjoy.py and train.py which are both broken by an earlier commit bb40378. These files relied on metadata added to the info object by SimpleMonitor, and the commit removes SimpleMonitor and replaces to call to it with another monitor which doesn't add the necessary info fields. As well, there is some deprecated module removed which was still referenced by these files.

This fork restores SimpleMonitor, adds it to the env, and changes the wrapper calls to current code (away from the deprecated/removed module).

Additional changes

• Updated ACKTR's run_atari.py to save its model when done training, and optionally every X updates during training. Added ability to resume training if the desired intermediate model is copied to the parent directory of the logger and named "checkpoint_model" (e.g. move checkpoint15000 to /tmp/checkpoint_model and it will resume training there).
• Added an ACKTR enjoy_atari.py to view the trained model playing its game. There's an option to display what the model sees (a scaled down grayscale from the actual game) so you can determine if a fault in the model is due to missing information from the scaling/grayscale view it gets.

Baselines

OpenAI Baselines is a set of high-quality implementations of reinforcement learning algorithms.

These algorithms will make it easier for the research community to replicate, refine, and identify new ideas, and will create good baselines to build research on top of. Our DQN implementation and its variants are roughly on par with the scores in published papers. We expect they will be used as a base around which new ideas can be added, and as a tool for comparing a new approach against existing ones.

You can install it by typing:

git clone https://github.com/openai/baselines.git
cd baselines
pip install -e .

• A2C
• ACER
• ACKTR
• DDPG
• DQN
• PPO1 (Multi-CPU using MPI)
• PPO2 (Optimized for GPU)
• TRPO

To cite this repository in publications:

@misc{baselines,
  author = {Dhariwal, Prafulla and Hesse, Christopher and Plappert, Matthias and Radford, Alec and Schulman, John and Sidor, Szymon and Wu, Yuhuai},
  title = {OpenAI Baselines},
  year = {2017},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/openai/baselines}},
}