From d0c1a87faf8490a8bdaa1ccd79da15cbf105ee81 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Marco=20Tr=C3=B6ster?= Date: Thu, 2 Feb 2023 12:34:38 +0100 Subject: [PATCH] Add scaling section to A2C documentation (#1250) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * add scaling section to A2C documentation * add cross-reference to vectorized envs article * turn it as note * update changelog * add Bonifatius94 to the list of contributors * fix issue number --------- Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com> Co-authored-by: Quentin GALLOUÉDEC Co-authored-by: Antonin RAFFIN --- docs/misc/changelog.rst | 3 ++- docs/modules/a2c.rst | 18 ++++++++++++++++++ 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/docs/misc/changelog.rst b/docs/misc/changelog.rst index 7ff0c0e..9dee356 100644 --- a/docs/misc/changelog.rst +++ b/docs/misc/changelog.rst @@ -39,6 +39,7 @@ Others: Documentation: ^^^^^^^^^^^^^^ - Renamed ``load_parameters`` to ``set_parameters`` (@DavyMorgan) +- Clarified documentation about subproc multiprocessing for A2C (@Bonifatius94) - Fixed typo in ``A2C`` docstring (@AlexPasqua) @@ -1222,4 +1223,4 @@ And all the contributors: @Gregwar @ycheng517 @quantitative-technologies @bcollazo @git-thor @TibiGG @cool-RR @MWeltevrede @Melanol @qgallouedec @francescoluciano @jlp-ue @burakdmb @timothe-chaumont @honglu2875 @yuanmingqi @anand-bala @hughperkins @sidney-tio @AlexPasqua @dominicgkerr @Akhilez @Rocamonde @tobirohrer @ZikangXiong -@DavyMorgan @luizapozzobon +@DavyMorgan @luizapozzobon @Bonifatius94 diff --git a/docs/modules/a2c.rst b/docs/modules/a2c.rst index e871424..670da61 100644 --- a/docs/modules/a2c.rst +++ b/docs/modules/a2c.rst @@ -76,6 +76,24 @@ Train a A2C agent on ``CartPole-v1`` using 4 environments. env.render() +.. note:: + + A2C is meant to be run primarily on the CPU, especially when you are not using a CNN. To improve CPU utilization, try turning off the GPU and using ``SubprocVecEnv`` instead of the default ``DummyVecEnv``: + + .. code-block:: + + from stable_baselines3 import A2C + from stable_baselines3.common.env_util import make_vec_env + from stable_baselines3.common.vec_env import SubprocVecEnv + + if __name__=="__main__": + env = make_vec_env("CartPole-v1", n_envs=8, vec_env_cls=SubprocVecEnv) + model = A2C("MlpPolicy", env, device="cpu") + model.learn(total_timesteps=25_000) + + For more information, see :ref:`Vectorized Environments `, `Issue #1245 `_ or the `Multiprocessing notebook `_. + + Results -------