* Modified ActorCriticPolicy to support non-shared features extractor
* Refactored features extraction with non-shared features extractor in ActorCriticPolicy and updated doc
Doc update: added 'warning' on custom policy docs that says that, if the features extractor is non-shared, it's not possible to have shared layers in the mlp_extractor
* Moved attrib share_features_extractor in class
* Updated custom policy doc for non-shared features extractor
* Updated changelog
* Made some if-statements more readable if policies.py
The if-statements are related to the shared/non-shared features extractor in ActorCritic policies
* Simplify implementation and add run test
* Keep order in module gain to keep previous results consistents
* Fix test
* Improved docstring in policies.py
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
* Added some tests
* feature extractor -> features extractor
* Fix test
* Fix env_id in test
* Make features extractor parameter explicit
* Remove duplicate
Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Co-authored-by: Antonin Raffin <antonin.raffin@dlr.de>
* Fix support of image like normalized inputs
* Improve docstring and warning message.
* Don't check if obs is image when normalize_images is False (lil opt)
* Comment fix
* Fix normalize_images not passed to parent
* Check for subclasses too
* Remove useless multiline
* Update version and add comment
* Fix some typos
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
* Replace .to(device) when possible
* fix numpy dep
* black
* Add warning for device != cpu and copy=False
* Update changelog
* Remove warning
* Update buffers.py
* Update version
* Fix type checking
Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
* Replace .to(device) when possible
* fix numpy dep
* black
* Add warning for device != cpu and copy=False
* Update changelog
* Remove warning
* Update buffers.py
* Updated custom policy docs
Better explained how the dimensions of the mlp_extractor work, including the action net and the value net after the layers specified in net_arch.
* Improved custom policy doc
Section: Custom Network Architecture.
Explained with greater detail that an action net and a value net will be added on top of the net_arch.
* Improved custom policy doc
Section: Custom Network Architecture.
Merged a comment into a note
* Alignment
Co-authored-by: Quentin GALLOUÉDEC <gallouedec.quentin@gmail.com>
* Add with_bias arg
* Update changelog
* move torch_layers to the last position
* Update version
Co-authored-by: Antonin Raffin <antonin.raffin@ensta.org>
* Raise error when same env object instance is passed in vectorized environment
* At to changelog
* Add raises to docstring
* Add test
* Also test make_vec_env
* Fix test
* Try to enable color for MyPy
* Update version and ignore lint warnings
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
Co-authored-by: Antonin Raffin <antonin.raffin@dlr.de>
* Install and configure mypy
* Test if github CI uses setup.cfg for mypy
* force color output
* tab to space
* Try to fix regex
* follow_imports silent
* use space as indentation
* fix indentation setup.cfg
* Show error code
* Update doc
* Udate changelog
* Ignore mypy cache files from commit
* Update gitlab CI
* Add pytype and mypy entry in Makefile
* Make mypy happy
Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
* Fixed errors in the documentation
Fixed grammatical and punctuation errors, and improved the sentence structure.
* Added username in the contributors
* Add PolicyPredictor protocol and use it in evaluate_policy
* Update changelog
* Move Protocol to type_aliases to avoid circular import
* Add test for evaluate_policy on BasePolicy
* Remove unused import
* Use typing_extensions
* Move typing_extensions to 3rd party
* Add version range (typing_extensions uses SemVer)
* Import Protocol from typing_extensions only on Python<3.8
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
* Install typing_extensions only on Python<3.8
* Add missing sys import
* Fix import ordering
* Fix observation type hint in predict
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Co-authored-by: Quentin GALLOUÉDEC <gallouedec.quentin@gmail.com>
* Adds deprecation warning if `eval_env` or `eval_freq` parameters are used. See #925
* added changelog entry
* added missing backtick
* deprecating `create_eval_env` parameter as well and adding comments to explain the `stacklevel` parameter used
* Updated tests to ignore DeprecationWarnings
* Updated changelog entry
* - Removed the `create_eval_env` parameter from the examples in the docs
- Removed information about the `create_eval_env` parameter from the migration docs
- Added information about deprecation of the `create_eval_env` parameter in the docs
* Add alternative in docstring
* Update docstrings
* `eval_freq` warning in docstring
* Add deprecation comments in tests
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
Co-authored-by: Quentin GALLOUÉDEC <gallouedec.quentin@gmail.com>
* Add progress bar callback and argument
* Update doc
* Update changelog
* Upgrade pytype in docker image
* Use tqdm.write in the logger to have cleaner output
* Fix logger test
* Fix when doing multiple calls to learn()
* Address comments from code-review
* Updated type hint and extended docstring in make_vec_env
The function itself was already working with callables, but it wasn't considerent in the type hint of the function's signature.
Extended the description of the wrapper_class parameter with a link to a Github issue containing more details on the matter.
* Updated type hint in make_atari_env
The function itself was already working with callables, but it wasn't considerent in the type hint of the function's signature.
* Updated docstring in make_atari_env
When modifying the type hint of the parameter 'env_id' (in this commit: fda6872f73c11075901ba88f2520f6316f818d1d), I forgot to update its description in the docstrig.
Doing it now.
* Removed redundant type in env_id's type hint in make_vec_env and make_atari_env
Callable[..., gym.Env] already includes Type[gym.Env], as pointed out here: https://github.com/DLR-RM/stable-baselines3/pull/1085#issuecomment-1269685218
Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
* Updated docstring from n_steps to n_rollout_steps
This must be a typo
* Fixed typo in a comment in ppo.py
* Update changelog
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Co-authored-by: Antonin Raffin <antonin.raffin@dlr.de>
* Fix return type for load, learn in BaseAlgorithm
* Update changelog
* Add typing extensions to dependencies
* Import directly from typing for python >3.11
* Reorder changelog to reflect merge order
* Roll back to typevar solution
* Updated changelog
* Remove typing extensions requirement
* Update base_class.py
* Remove final point in changelog
* Additional type fixes across project
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
* Fix loading with new `n_envs`
* Update tests
* Update changelog
* Fix the fix
* Remove `self._setup_model()` from `set_env()`
* Raise `AssertionError` when setting env with a different `n_envs`
* Update unitests
Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
