Update SB3 contrib doc (+ fix backward compat) (#707)

* Fix `VecNormalize` load for SB3<= 1.3.0

* Update SB3 contrib doc

* Bump version

README.md

@@ -165,10 +165,11 @@ All the following examples can be executed online using Google colab notebooks:
 | DQN   | :x: | :x: | :heavy_check_mark: | :x:                 | :x:                | :heavy_check_mark: |
 | HER   | :x: | :heavy_check_mark: | :heavy_check_mark: | :x:                 | :x:                | :x: |
 | PPO   | :x: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark:  | :heavy_check_mark: | :heavy_check_mark: |
+| QR-DQN<sup>[1](#f1)</sup>  | :x: | :x: | :heavy_check_mark: | :x:                 | :x:                | :heavy_check_mark: |
 | SAC   | :x: | :heavy_check_mark: | :x:                | :x:                 | :x:                | :heavy_check_mark: |
 | TD3   | :x: | :heavy_check_mark: | :x:                | :x:                 | :x:                | :heavy_check_mark: |
-| QR-DQN<sup>[1](#f1)</sup>  | :x: | :x: | :heavy_check_mark: | :x:                 | :x:                | :heavy_check_mark: |
 | TQC<sup>[1](#f1)</sup>   | :x: | :heavy_check_mark: | :x:                | :x:                 | :x: | :heavy_check_mark: |
+| TRPO<sup>[1](#f1)</sup>  | :x: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark:  | :heavy_check_mark: | :heavy_check_mark: |
 | Maskable PPO<sup>[1](#f1)</sup>   | :x: | :x: | :heavy_check_mark: | :heavy_check_mark:  | :heavy_check_mark: | :heavy_check_mark:  |
 
 <b id="f1">1</b>: Implemented in [SB3 Contrib](https://github.com/Stable-Baselines-Team/stable-baselines3-contrib) GitHub repository.

docs/guide/algos.rst

@@ -13,10 +13,11 @@ DDPG                 ✔️           ❌            ❌                ❌
 DQN                  ❌           ✔️            ❌                ❌               ✔️
 HER                  ✔️           ✔️            ❌                ❌               ❌
 PPO                  ✔️           ✔️            ✔️                 ✔️               ✔️
+QR-DQN [#f1]_        ❌          ️ ✔️            ❌                ❌               ✔️
 SAC                  ✔️           ❌            ❌                ❌               ✔️
 TD3                  ✔️           ❌            ❌                ❌               ✔️
-QR-DQN [#f1]_        ❌          ️ ✔️            ❌                ❌               ✔️
 TQC [#f1]_           ✔️           ❌            ❌                ❌               ✔️
+TRPO  [#f1]_         ✔️           ✔️            ✔️                 ✔️               ✔️
 Maskable PPO [#f1]_  ❌           ✔️            ✔️                 ✔️               ✔️
 ===================  =========== ============ ================= =============== ================
 

docs/guide/sb3_contrib.rst

@@ -8,7 +8,7 @@ We implement experimental features in a separate contrib repository:
 `SB3-Contrib`_
 
 This allows Stable-Baselines3 (SB3) to maintain a stable and compact core, while still
-providing the latest features, like Truncated Quantile Critics (TQC) or
+providing the latest features, like Truncated Quantile Critics (TQC), Trust Region Policy Optimization (TRPO) or
 Quantile Regression DQN (QR-DQN).
 
 Why create this repository?
@@ -36,8 +36,9 @@ See documentation for the full list of included features.
 
 **RL Algorithms**:
 
-- `Truncated Quantile Critics (TQC)`_
 - `Quantile Regression DQN (QR-DQN)`_
+- `Truncated Quantile Critics (TQC)`_
+- `Trust Region Policy Optimization (TRPO) <https://arxiv.org/abs/1502.05477>`_
 - `PPO with invalid action masking (Maskable PPO) <https://arxiv.org/abs/2006.14171>`_
 
 **Gym Wrappers**:

docs/misc/changelog.rst

@@ -4,7 +4,7 @@ Changelog
 ==========
 
 
-Release 1.3.1a7 (WIP)
+Release 1.3.1a8 (WIP)
 ---------------------------
 
 Breaking Changes:
@@ -53,7 +53,7 @@ Documentation:
 - Added link to RL Tips and Tricks video
 - Updated ``BaseAlgorithm.load`` docstring (@Demetrio92)
 - Added a note on ``load`` behavior in the examples (@Demetrio92)
-
+- Updated SB3 Contrib doc
 
 Release 1.3.0 (2021-10-23)
 ---------------------------

stable_baselines3/common/vec_env/vec_normalize.py

@@ -117,7 +117,7 @@ class VecNormalize(VecEnvWrapper):
 
         :param state:"""
         # Backward compatibility
-        if "norm_obs_keys" not in state:
+        if "norm_obs_keys" not in state and isinstance(state["observation_space"], gym.spaces.Dict):
             state["norm_obs_keys"] = list(state["observation_space"].spaces.keys())
         self.__dict__.update(state)
         assert "venv" not in state

stable_baselines3/version.txt

@@ -1 +1 @@
-1.3.1a7
+1.3.1a8