Update doc

This commit is contained in:
Antonin RAFFIN 2020-05-08 13:09:38 +02:00
parent 97aea21349
commit 8a61913a1d
11 changed files with 66 additions and 17 deletions

View file

@ -0,0 +1,7 @@
.. _atari_wrapper:
Atari Wrappers
==============
.. automodule:: stable_baselines3.common.atari_wrappers
:members:

View file

@ -0,0 +1,7 @@
.. _cmd_utils:
Command Utils
=========================
.. automodule:: stable_baselines3.common.cmd_util
:members:

7
docs/common/logger.rst Normal file
View file

@ -0,0 +1,7 @@
.. _logger:
Logger
======
.. automodule:: stable_baselines3.common.logger
:members:

7
docs/common/monitor.rst Normal file
View file

@ -0,0 +1,7 @@
.. _monitor:
Monitor Wrapper
===============
.. automodule:: stable_baselines3.common.monitor
:members:

7
docs/common/noise.rst Normal file
View file

@ -0,0 +1,7 @@
.. _noise:
Action Noise
=============
.. automodule:: stable_baselines3.common.noise
:members:

7
docs/common/utils.rst Normal file
View file

@ -0,0 +1,7 @@
.. _utils:
Utils
=====
.. automodule:: stable_baselines3.common.utils
:members:

View file

@ -39,7 +39,7 @@ however, *don't expect the default ones to work* on any environment.
Therefore, we *highly recommend you* to take a look at the `RL zoo <https://github.com/DLR-RM/rl-baselines3-zoo>`_ (or the original papers) for tuned hyperparameters. Therefore, we *highly recommend you* to take a look at the `RL zoo <https://github.com/DLR-RM/rl-baselines3-zoo>`_ (or the original papers) for tuned hyperparameters.
A best practice when you apply RL to a new problem is to do automatic hyperparameter optimization. Again, this is included in the `RL zoo <https://github.com/DLR-RM/rl-baselines3-zoo>`_. A best practice when you apply RL to a new problem is to do automatic hyperparameter optimization. Again, this is included in the `RL zoo <https://github.com/DLR-RM/rl-baselines3-zoo>`_.
When applying RL to a custom problem, you should always normalize the input to the agent (e.g. using VecNormalize for PPO2/A2C) When applying RL to a custom problem, you should always normalize the input to the agent (e.g. using VecNormalize for PPO/A2C)
and look at common preprocessing done on other environments (e.g. for `Atari <https://danieltakeshi.github.io/2016/11/25/frame-skipping-and-preprocessing-for-deep-q-networks-on-atari-2600-games/>`_, frame-stack, ...). and look at common preprocessing done on other environments (e.g. for `Atari <https://danieltakeshi.github.io/2016/11/25/frame-skipping-and-preprocessing-for-deep-q-networks-on-atari-2600-games/>`_, frame-stack, ...).
Please refer to *Tips and Tricks when creating a custom environment* paragraph below for more advice related to custom environments. Please refer to *Tips and Tricks when creating a custom environment* paragraph below for more advice related to custom environments.
@ -137,7 +137,7 @@ Please use the hyperparameters in the `RL zoo <https://github.com/DLR-RM/rl-base
Continuous Actions - Multiprocessed Continuous Actions - Multiprocessed
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Take a look at PPO2, TRPO or A2C. Again, don't forget to take the hyperparameters from the `RL zoo <https://github.com/DLR-RM/rl-baselines3-zoo>`_ Take a look at PPO, TRPO or A2C. Again, don't forget to take the hyperparameters from the `RL zoo <https://github.com/DLR-RM/rl-baselines3-zoo>`_
for continuous actions problems (cf *Bullet* envs). for continuous actions problems (cf *Bullet* envs).
.. note:: .. note::

View file

@ -3,11 +3,11 @@
You can adapt this file completely to your liking, but it should at least You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive. contain the root `toctree` directive.
Welcome to Stable Baselines3 docs! Welcome to Stable Baselines3 docs! - RL Baselines Made Easy
================================== ===========================================================
`Stable Baselines3 <https://github.com/DLR-RM/stable-baselines3>`_ is a set of improved implementations of reinforcement learning algorithms in PyTorch. `Stable Baselines3 <https://github.com/DLR-RM/stable-baselines3>`_ is a set of improved implementations of reinforcement learning algorithms in PyTorch.
It is the next major version (PyTorch edition) of `Stable Baselines <https://github.com/hill-a/stable-baselines>`_. It is the next major version of `Stable Baselines <https://github.com/hill-a/stable-baselines>`_.
Github repository: https://github.com/DLR-RM/stable-baselines3 Github repository: https://github.com/DLR-RM/stable-baselines3
@ -61,9 +61,15 @@ Main Features
:maxdepth: 1 :maxdepth: 1
:caption: Common :caption: Common
common/atari_wrappers
common/cmd_utils
common/distributions common/distributions
common/evaluation common/evaluation
common/env_checker common/env_checker
common/monitor
common/logger
common/noise
common/utils
.. toctree:: .. toctree::
:maxdepth: 1 :maxdepth: 1

View file

@ -33,10 +33,11 @@ class AtariWrapper(gym.Wrapper):
:param screen_size: (int): resize Atari frame :param screen_size: (int): resize Atari frame
:param terminal_on_life_loss: (bool): if True, then step() returns done=True whenever a :param terminal_on_life_loss: (bool): if True, then step() returns done=True whenever a
life is lost. life is lost.
:param grayscale_obs: (bool): if True, then gray scale observation is returned, otherwise, RGB observation :param grayscale_obs: (bool): if True (default), then gray scale observation is returned, otherwise, RGB observation
is returned. is returned.
:param scale_obs: (bool): if True, then observation normalized in range [0,1] is returned. It also limits memory :param scale_obs: (bool): if True, then observation normalized in range [0,1] is returned. It also limits memory
optimization benefits of FrameStack Wrapper. optimization benefits of FrameStack Wrapper.
:param scale_obs: (bool) If True (default), the reward is clip to {-1, 0, 1} depending on its sign.
""" """
def __init__(self, env: gym.Env, def __init__(self, env: gym.Env,
noop_max: int = 30, noop_max: int = 30,

View file

@ -94,7 +94,7 @@ def make_atari_env(env_id: Union[str, Type[gym.Env]],
:param monitor_dir: (str) Path to a folder where the monitor files will be saved. :param monitor_dir: (str) Path to a folder where the monitor files will be saved.
If None, no file will be written, however, the env will still be wrapped If None, no file will be written, however, the env will still be wrapped
in a Monitor wrapper to provide additional information about training. in a Monitor wrapper to provide additional information about training.
:param wrapper_kwargs: (Dict[str, Any]) Optional keyword argument to pass to the env constructor :param wrapper_kwargs: (Dict[str, Any]) Optional keyword argument to pass to the ``AtariWrapper``
:param env_kwargs: (Dict[str, Any]) Optional keyword argument to pass to the env constructor :param env_kwargs: (Dict[str, Any]) Optional keyword argument to pass to the env constructor
:param vec_env_cls: (Type[VecEnv]) A custom `VecEnv` class constructor. Default: None. :param vec_env_cls: (Type[VecEnv]) A custom `VecEnv` class constructor. Default: None.
:param vec_env_kwargs: (Dict[str, Any]) Keyword arguments to pass to the `VecEnv` class constructor. :param vec_env_kwargs: (Dict[str, Any]) Keyword arguments to pass to the `VecEnv` class constructor.

View file

@ -13,6 +13,16 @@ import numpy as np
class Monitor(gym.Wrapper): class Monitor(gym.Wrapper):
"""
A monitor wrapper for Gym environments, it is used to know the episode reward, length, time and other data.
:param env: (gym.Env) The environment
:param filename: (Optional[str]) the location to save a log file, can be None for no log
:param allow_early_resets: (bool) allows the reset of the environment before it is done
:param reset_keywords: (Tuple[str, ...]) extra keywords for the reset call,
if extra parameters are needed at reset
:param info_keywords: (Tuple[str, ...]) extra information to log, from the information return of env.step()
"""
EXT = "monitor.csv" EXT = "monitor.csv"
def __init__(self, def __init__(self,
@ -21,16 +31,6 @@ class Monitor(gym.Wrapper):
allow_early_resets: bool = True, allow_early_resets: bool = True,
reset_keywords: Tuple[str, ...] = (), reset_keywords: Tuple[str, ...] = (),
info_keywords: Tuple[str, ...] = ()): info_keywords: Tuple[str, ...] = ()):
"""
A monitor wrapper for Gym environments, it is used to know the episode reward, length, time and other data.
:param env: (gym.Env) The environment
:param filename: (Optional[str]) the location to save a log file, can be None for no log
:param allow_early_resets: (bool) allows the reset of the environment before it is done
:param reset_keywords: (Tuple[str, ...]) extra keywords for the reset call,
if extra parameters are needed at reset
:param info_keywords: (Tuple[str, ...]) extra information to log, from the information return of env.step()
"""
super(Monitor, self).__init__(env=env) super(Monitor, self).__init__(env=env)
self.t_start = time.time() self.t_start = time.time()
if filename is None: if filename is None: