more verbose documentation regarding .load vs .set_parameters (#696)

* more verbose documentation regarding `.load` vs `.set_parameters` (#683, #614)

* add a note to explain the difference between `.load` and `.set_parameters` to the examples

* fix typos

Co-authored-by: Anssi <kaneran21@hotmail.com>

Co-authored-by: Anssi <kaneran21@hotmail.com>

docs/guide/examples.rst

@@ -56,10 +56,11 @@ In the following example, we will train, save and load a DQN model on the Lunar
   LunarLander requires the python package ``box2d``.
   You can install it using ``apt install swig`` and then ``pip install box2d box2d-kengz``
 
-.. .. note::
-..   ``load`` function re-creates model from scratch on each call, which can be slow.
-..   If you need to e.g. evaluate same model with multiple different sets of parameters, consider
-..   using ``load_parameters`` instead.
+.. note::
+  ``load`` method re-creates the model from scratch and should be called on the Algorithm without instantiating it first,
+  e.g. ``model = DQN.load("dqn_lunar", env=env)`` instead of ``model = DQN(env=env)`` followed by  ``model.load("dqn_lunar")``. The latter **will not work** as ``load`` does not work by reference.
+  If you want to load parameters without re-creating the model, e.g. to evaluate the same model 
+  with multiple different sets of parameters, consider using ``set_parameters`` instead.
 
 .. code-block:: python
 

docs/misc/changelog.rst

@@ -49,6 +49,9 @@ Documentation:
 - Add documentation on exporting to TFLite/Coral
 - Added JMLR paper and updated citation
 - Added link to RL Tips and Tricks video
+- Update ``BaseAlgorithm.load`` docstring
+- Add a Note on ``load`` behavior in the examples
+
 
 Release 1.3.0 (2021-10-23)
 ---------------------------

stable_baselines3/common/base_class.py

@@ -657,7 +657,9 @@ class BaseAlgorithm(ABC):
         **kwargs,
     ) -> "BaseAlgorithm":
         """
-        Load the model from a zip-file
+        Load the model from a zip-file.
+        Note: `load` re-creates the model from scratch, it does not update it in-place!
+        For an in-place load use `set_parameters` instead.
 
         :param path: path to the file (or a file-like) where to
             load the agent from
@@ -676,6 +678,7 @@ class BaseAlgorithm(ABC):
             to avoid unexpected behavior.
             See https://github.com/DLR-RM/stable-baselines3/issues/597
         :param kwargs: extra arguments to change the model when loading
+        :return: new model instance with loaded parameters
         """
         if print_system_info:
             print("== CURRENT SYSTEM INFO ==")