* Fix reward of SimpleMultiObsEnv to always be float
Previously the reward was sometimes returned as an int.
* changelog
* Update changelog.rst
* Update version.txt
* Fix type annotation
* Fix import
---------
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Co-authored-by: Antonin Raffin <antonin.raffin@ensta.org>
* prevents squash_output if not use_sde, see #1592
* update changelog
* add unscaling of actions taken during training
* add test regarding squashing and unquashing
* avoids try-except block
* format Gymnasium code with black
* makes mypy pass
* makes pytype pass
* sort imports
* makes error message in assert statement clearer
Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
* improves code commenting
* replaces full env with wrapper
* Cleanup code
* Reformat
---------
Co-authored-by: PatrickHelm <patrick.helm@gmx.net>
Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
Co-authored-by: Antonin Raffin <antonin.raffin@dlr.de>
* Fix type hints in `common/utils.py`
* Fix `VecTranspose` type annotations
* Fix types for callbacks
* Update changelog
* Fix video recorder type hints
* Fix save utils type hints
* Allow BytesIO
* Improve error message
* Make logger and training env properties
* Clarify which open_path fn is called
* Fix bug in env_checker.py bounds warning message
* Fix bug where Gym Environment Checker does not output the correct warning message when dealing with observation spaces that have different upper and different lower bounds
* Update test_env_checker.py with more comprehensive tests
* Make naming consistent
* Update version
* Catch all invalid indices at once
---------
Co-authored-by: gabo_tor <gabriel0torre@gmail.com>
* Added test cases where off policy algorithms fail with float64 actionspace
* casting observations and actions to `np.float32` to unify behaviour between `ReplayBuffer` and `RolloutBuffer`. Fixing issue #1145
* reformatted using black
* making test more restrictive by checking models action is float64
* added changelog entry
* undo cast of observations as `preprocessing.preprocess_obs()` casts them to float32 anyways.
* - Casting to float32 only, if action.dtype is float64
- Added cast to `DictReplayBuffer` as well
* Added tests for multiple variations of continuous action types and observation spaces
* applied reformatting by `make commit-checks`
* Added typing and comment referring to description in merge request
* Apply linter for single element slice
* Rename helper and refactor tests
* Update changelog and docstring
---------
Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
* Docs: Add mobile-env to community projects
* Update docs Readme with correct install command
Without the quotes, I get `no matches found: .[docs]`
* Add changelog entry for adding mobile-env
* Fix format in projects.rst
Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
---------
Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
* Update setup.py to v0.29.0
* Remove invalid test
* Loosen version and update changelog
---------
Co-authored-by: Antonin Raffin <antonin.raffin@ensta.org>
* change ordering of achieved_goal and desired_goal to match expected compute_reward order
* Update changelog.rst
* Update version
* Update version.txt
* Update changelog.rst
---------
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
* Fix env checker single-step-env edge case
Before this change, env checker failed to `reset()` the tested
environment before calling `step()` when checking for `Inf` / `NaN`.
This could cause environments which happened to have only one `step()`
available before the episode was terminated to fail.
This is now fixed.
* Code review fixes#1
As suggested by Antonin Raffin <antonin.raffin@ensta.org>.
* Switch from List to Sequence for `seed()` type hint
* Fix logger type hints
* Improve replay buffer type hints
* Fix custom envs type annotations
* Fix VecMonitor type hints
* Fix RMSprop type hint
* Fix vec extract dict obs type hints
* Fix vec frame stack type annotations
* Fix base vec env type hints
* Fix dummy vec env type hints
* Fix for mypy
* Fixes for the tests
* mypy doesn't like when we overwrite type
* fix step of SimpleMultiObsEnv
* remove useless type specification
* Rm useless type hint
* Improve logger type hint
* format
* rm useless type hint
* Re-add variables in constructor, remove unused import
---------
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
* Moved 'Monitoring Training' to subsubsection of 'Using callbacks'
* Added EvalCallback example
* Updated Changelogs
* Edited the language
* Moved subsection headers up one level
* added make_vec_env into Evalcallback example
* Added parameters to the top for readability
* Added note on multiple training environments
* Added more clarity to eval_freq note
* Apply suggestions from code review
---------
Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
* Fix failing set_env test
* Fix test failiing due to deprectation of env.seed
* Adjust mean reward threshold in failing test
* Fix her test failing due to rng
* Change seed and revert reward threshold to 90
* Pin gym version
* Make VecEnv compatible with gym seeding change
* Revert change to VecEnv reset signature
* Change subprocenv seed cmd to call reset instead
* Fix type check
* Add backward compat
* Add `compat_gym_seed` helper
* Add goal env checks in env_checker
* Add docs on HER requirements for envs
* Capture user warning in test with inverted box space
* Update ale-py version
* Fix randint
* Allow noop_max to be zero
* Update changelog
* Update docker image
* Update doc conda env and dockerfile
* Custom envs should not have any warnings
* Fix test for numpy >= 1.21
* Add check for vectorized compute reward
* Bump to gym 0.24
* Fix gym default step docstring
* Test downgrading gym
* Revert "Test downgrading gym"
This reverts commit 0072b77156c006ada8a1d6e26ce347ed85a83eeb.
* Fix protobuf error
* Fix in dependencies
* Fix protobuf dep
* Use newest version of cartpole
* Update gym
* Fix warning
* Loosen required scipy version
* Scipy no longer needed
* Try gym 0.25
* Silence warnings from gym
* Filter warnings during tests
* Update doc
* Update requirements
* Add gym 26 compat in vec env
* Fixes in envs and tests for gym 0.26+
* Enforce gym 0.26 api
* format
* Fix formatting
* Fix dependencies
* Fix syntax
* Cleanup doc and warnings
* Faster tests
* Higher budget for HER perf test (revert prev change)
* Fixes and update doc
* Fix doc build
* Fix breaking change
* Fixes for rendering
* Rename variables in monitor
* update render method for gym 0.26 API
backwards compatible (mode argument is allowed) while using the gym 0.26 API (render mode is determined at environment creation)
* update tests and docs to new gym render API
* undo removal of render modes metatadata check
* set rgb_array as default render mode for gym.make
* undo changes & raise warning if not 'rgb_array'
* Fix type check
* Remove recursion and fix type checking
* Remove hacks for protobuf and gym 0.24
* Fix type annotations
* reuse existing render_mode attribute
* return tiled images for 'human' render mode
* Allow to use opencv for human render, fix typos
* Add warning when using non-zero start with Discrete (fixes#1197)
* Fix type checking
* Bug fixes and handle more cases
* Throw proper warnings
* Update test
* Fix new metadata name
* Ignore numpy warnings
* Fixes in vec recorder
* Global ignore
* Filter local warning too
* Monkey patch not needed for gym 26
* Add doc of VecEnv vs Gym API
* Add render test
* Fix return type
* Update VecEnv vs Gym API doc
* Fix for custom render mode
* Fix return type
* Fix type checking
* check test env test_buffer
* skip render check
* check env test_dict_env
* test_env test_gae
* check envs in remaining tests
* Update tests
* Add warning for Discrete action space with non-zero (#1295)
* Fix atari annotation
* ignore get_action_meanings [attr-defined]
* Fix mypy issues
* Add patch for gym/gymnasium transition
* Switch to gymnasium
* Rely on signature instead of version
* More patches
* Type ignore because of https://github.com/Farama-Foundation/Gymnasium/pull/39
* Fix doc build
* Fix pytype errors
* Fix atari requirement
* Update env checker due to change in dtype for Discrete
* Fix type hint
* Convert spaces for saved models
* Ignore pytype
* Remove gitlab CI
* Disable pytype for convert space
* Fix undefined info
* Fix undefined info
* Upgrade shimmy
* Fix wrappers type annotation (need PR from Gymnasium)
* Fix gymnasium dependency
* Fix dependency declaration
* Cap pygame version for python 3.7
* Point to master branch (v0.28.0)
* Fix: use main not master branch
* Rename done to terminated
* Fix pygame dependency for python 3.7
* Rename gym to gymnasium
* Update Gymnasium
* Fix test
* Fix tests
* Forks don't have access to private variables
* Fix linter warnings
* Update read the doc env
* Fix env checker for GoalEnv
* Fix import
* Update env checker (more info) and fix dtype
* Use micromamab for Docker
* Update dependencies
* Clarify VecEnv doc
* Fix Gymnasium version
* Copy file only after mamba install
* [ci skip] Update docker doc
* Polish code
* Reformat
* Remove deprecated features
* Ignore warning
* Update doc
* Update examples and changelog
* Fix type annotation bundle (SAC, TD3, A2C, PPO, base class) (#1436)
* Fix SAC type hints, improve DQN ones
* Fix A2C and TD3 type hints
* Fix PPO type hints
* Fix on-policy type hints
* Fix base class type annotation, do not use defaults
* Update version
* Disable mypy for python 3.7
* Rename Gym26StepReturn
* Update continuous critic type annotation
* Fix pytype complain
---------
Co-authored-by: Carlos Luis <carlos.luisgonc@gmail.com>
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Co-authored-by: Thomas Lips <37955681+tlpss@users.noreply.github.com>
Co-authored-by: tlips <thomas.lips@ugent.be>
Co-authored-by: tlpss <thomas17.lips@gmail.com>
Co-authored-by: Quentin GALLOUÉDEC <gallouedec.quentin@gmail.com>
* VecExtractDictObs handle terminal_observation
* Added VecExtractDictObs handle terminal_output to changelog
* Update changelog.rst
* Update test_vec_extract_dict_obs.py
Add random dones in env to test if terminal_observation is properly handled
* Made test deterministic
* Fixed bug in test
* Improved test
* Fix format in test
* Update test
* Fix type hint
* Ignore pytype warning
* Ignore pytype
---------
Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
* Rename the observations variable in the evaluation util to avoid shadowing
This enables a callback in evaluate_policy to have access to the
observation vector that is fed to the environment step function,
which is currently shadowed by the output observation.
* Update changelog
* Add test
* Move assignment outside of the loop
---------
Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
Co-authored-by: Antonin Raffin <antonin.raffin@dlr.de>
* Fix type hints for DQN
* [ci skip] Remove commented line
* Refine types
* Fix vectorized obs detection
* Fix for pytype
* Fix check at load time to create replay buffer
* One config file to rule them all
* Delete unused config
---------
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
* add instructions for running single tests in the README, add assertions for observation_space
* update changelog
* address linting warnings
* correct pytest command in the README
* correct review comments, run make commit-checks
* truncate lines that are too long
* address make lint warning about checking module availability
* fix tests
* use f-strings for formatting assertion messages
* fix type issue
* Refactor tests, improve error messages
---------
Co-authored-by: Antonin Raffin <antonin.raffin@dlr.de>
