From f13de5b57b05a539077f629b82493d558fdfa41b Mon Sep 17 00:00:00 2001 From: Anssi Date: Fri, 5 Mar 2021 18:05:14 +0200 Subject: [PATCH] Add note on loading and resetting environments (#340) * Update documentation and changelog * Fix docs * Update examples.rst Co-authored-by: Antonin RAFFIN --- docs/guide/examples.rst | 15 ++++++++++++--- docs/misc/changelog.rst | 1 + 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/docs/guide/examples.rst b/docs/guide/examples.rst index 539b2d4..1974d43 100644 --- a/docs/guide/examples.rst +++ b/docs/guide/examples.rst @@ -480,13 +480,22 @@ By default, the replay buffer is not saved when calling ``model.save()``, in ord However, SB3 provides a ``save_replay_buffer()`` and ``load_replay_buffer()`` method to save it separately. -.. image:: ../_static/img/colab-badge.svg - :target: https://colab.research.google.com/github/Stable-Baselines-Team/rl-colab-notebooks/blob/sb3/advanced_saving_loading.ipynb - Stable-Baselines3 automatic creation of an environment for evaluation. For that, you only need to specify ``create_eval_env=True`` when passing the Gym ID of the environment while creating the agent. Behind the scene, SB3 uses an :ref:`EvalCallback `. + +.. note:: + + For training model after loading it, we recommend loading the replay buffer to ensure stable learning (for off-policy algorithms). + You also need to pass ``reset_num_timesteps=True`` to ``learn`` function which initializes the environment + and agent for training if a new environment was created since saving the model. + + +.. image:: ../_static/img/colab-badge.svg + :target: https://colab.research.google.com/github/Stable-Baselines-Team/rl-colab-notebooks/blob/sb3/advanced_saving_loading.ipynb + + .. code-block:: python from stable_baselines3 import SAC diff --git a/docs/misc/changelog.rst b/docs/misc/changelog.rst index 484441a..fcde57e 100644 --- a/docs/misc/changelog.rst +++ b/docs/misc/changelog.rst @@ -14,6 +14,7 @@ Documentation: ^^^^^^^^^^^^^^ - Fixed examples - Added new project using SB3: rl_reach (@PierreExeter) +- Add a note on continual learning and resetting environment Pre-Release 0.11.1 (2021-02-27)