2019-09-26 09:46:40 +00:00
.. _changelog:
Changelog
==========
2020-11-02 09:43:25 +00:00
Pre-Release 0.11.0a0 (WIP)
-------------------------------
Breaking Changes:
^^^^^^^^^^^^^^^^^
New Features:
^^^^^^^^^^^^^
Bug Fixes:
^^^^^^^^^^
Deprecations:
^^^^^^^^^^^^^
Others:
^^^^^^^
- Add more issue templates
Documentation:
^^^^^^^^^^^^^^
- Updated algorithm table
2020-11-02 10:45:08 +00:00
- Minor docstring improvements regarding rollout (@stheid)
2020-11-02 09:43:25 +00:00
2020-10-07 08:51:49 +00:00
2020-10-28 12:01:56 +00:00
Pre-Release 0.10.0 (2020-10-28)
-------------------------------
**HER with online and offline sampling, bug fixes for features extraction**
2020-10-07 08:51:49 +00:00
Breaking Changes:
^^^^^^^^^^^^^^^^^
2020-10-22 09:05:52 +00:00
- **Warning:** Renamed `` common.cmd_util `` to `` common.env_util `` for clarity (affects `` make_vec_env `` and `` make_atari_env `` functions)
2020-10-07 08:51:49 +00:00
New Features:
^^^^^^^^^^^^^
2020-10-13 10:01:33 +00:00
- Allow custom actor/critic network architectures using `` net_arch=dict(qf=[400, 300], pi=[64, 64]) `` for off-policy algorithms (SAC, TD3, DDPG)
2020-10-22 09:56:43 +00:00
- Added Hindsight Experience Replay `` HER `` . (@megan-klaiber)
- `` VecNormalize `` now supports `` gym.spaces.Dict `` observation spaces
2020-10-22 09:33:58 +00:00
- Support logging videos to Tensorboard (@SwamyDev)
2020-10-27 13:24:59 +00:00
- Added `` share_features_extractor `` argument to `` SAC `` and `` TD3 `` policies
2020-10-07 08:51:49 +00:00
Bug Fixes:
^^^^^^^^^^
2020-10-12 21:10:54 +00:00
- Fix GAE computation for on-policy algorithms (off-by one for the last value) (thanks @Wovchena)
2020-10-22 09:56:43 +00:00
- Fixed potential issue when loading a different environment
2020-10-16 15:34:49 +00:00
- Fix ignoring the exclude parameter when recording logs using json, csv or log as logging format (@SwamyDev)
2020-10-16 09:09:19 +00:00
- Make `` make_vec_env `` support the `` env_kwargs `` argument when using an env ID str (@ManifoldFR)
2020-10-18 18:51:56 +00:00
- Fix model creation initializing CUDA even when `device="cpu"`
2020-10-18 21:23:51 +00:00
- Fix `` check_env `` not checking if the env has a Dict actionspace before calling `` _check_nan `` (@wmmc88)
- Update the check for spaces unsupported by Stable Baselines 3 to include checks on the action space (@wmmc88)
2020-10-27 13:24:59 +00:00
- Fixed feature extractor bug for target network where the same net was shared instead
of being separate. This bug affects `` SAC `` , `` DDPG `` and `` TD3 `` when using `` CnnPolicy `` (or custom feature extractor)
2020-10-27 21:12:52 +00:00
- Fixed a bug when passing an environment when loading a saved model with a `` CnnPolicy `` , the passed env was not wrapped properly
(the bug was introduced when implementing `` HER `` so it should not be present in previous versions)
2020-10-07 08:51:49 +00:00
Deprecations:
^^^^^^^^^^^^^
Others:
^^^^^^^
- Improved typing coverage
- Improved error messages for unsupported spaces
2020-10-16 09:09:19 +00:00
- Added `` .vscode `` to the gitignore
2020-10-07 08:51:49 +00:00
Documentation:
^^^^^^^^^^^^^^
2020-10-11 21:22:12 +00:00
- Added first draft of migration guide
2020-10-24 16:33:26 +00:00
- Added intro to `imitation <https://github.com/HumanCompatibleAI/imitation> `_ library (@shwang)
2020-10-22 09:56:43 +00:00
- Enabled doc for `` CnnPolicies ``
2020-10-28 08:55:16 +00:00
- Added advanced saving and loading example
- Added base doc for exporting models
- Added example for getting and setting model parameters
2020-10-07 08:51:49 +00:00
2020-10-04 15:12:35 +00:00
Pre-Release 0.9.0 (2020-10-03)
2020-08-05 10:12:02 +00:00
------------------------------
2020-10-04 15:12:35 +00:00
**Bug fixes, get/set parameters and improved docs**
2020-08-05 10:12:02 +00:00
Breaking Changes:
^^^^^^^^^^^^^^^^^
2020-08-23 11:27:52 +00:00
- Removed `` device `` keyword argument of policies; use `` policy.to(device) `` instead. (@qxcv)
2020-10-11 21:22:12 +00:00
- Rename `` BaseClass.get_torch_variables `` -> `` BaseClass._get_torch_save_params `` and `` BaseClass.excluded_save_params `` -> `` BaseClass._excluded_save_params ``
2020-09-24 14:05:36 +00:00
- Renamed saved items `` tensors `` to `` pytorch_variables `` for clarity
2020-09-29 17:41:14 +00:00
- `` make_atari_env `` , `` make_vec_env `` and `` set_random_seed `` must be imported with (and not directly from `` stable_baselines3.common `` ):
.. code-block :: python
from stable_baselines3.common.cmd_util import make_atari_env, make_vec_env
from stable_baselines3.common.utils import set_random_seed
2020-08-05 10:12:02 +00:00
New Features:
^^^^^^^^^^^^^
- Added `` unwrap_vec_wrapper() `` to `` common.vec_env `` to extract `` VecEnvWrapper `` if needed
2020-08-28 09:36:33 +00:00
- Added `` StopTrainingOnMaxEpisodes `` to callback collection (@xicocaio)
2020-09-20 17:13:18 +00:00
- Added `` device `` keyword argument to `` BaseAlgorithm.load() `` (@liorcohen5)
2020-09-10 14:56:27 +00:00
- Callbacks have access to rollout collection locals as in SB2. (@PartiallyTyped)
2020-09-24 12:28:27 +00:00
- Added `` get_parameters `` and `` set_parameters `` for accessing/setting parameters of the agent
2020-09-23 20:40:41 +00:00
- Added actor/critic loss logging for TD3. (@mloo3)
2020-08-05 10:12:02 +00:00
Bug Fixes:
^^^^^^^^^^
2020-10-22 09:56:43 +00:00
- Added `` unwrap_vec_wrapper() `` to `` common.vec_env `` to extract `` VecEnvWrapper `` if needed
2020-08-05 10:12:02 +00:00
- Fixed a bug where the environment was reset twice when using `` evaluate_policy ``
2020-09-01 07:52:31 +00:00
- Fix logging of `` clip_fraction `` in PPO (@diditforlulz273)
2020-09-20 17:13:18 +00:00
- Fixed a bug where cuda support was wrongly checked when passing the GPU index, e.g., `` device="cuda:0" `` (@liorcohen5)
2020-09-24 12:28:27 +00:00
- Fixed a bug when the random seed was not properly set on cuda when passing the GPU index
2020-08-05 10:12:02 +00:00
Deprecations:
^^^^^^^^^^^^^
Others:
^^^^^^^
- Improve typing coverage of the `` VecEnv ``
2020-09-23 08:34:35 +00:00
- Fix type annotation of `` make_vec_env `` (@ManifoldFR)
2020-08-05 10:12:02 +00:00
- Removed `` AlreadySteppingError `` and `` NotSteppingError `` that were not used
2020-08-23 15:44:35 +00:00
- Fixed typos in SAC and TD3
2020-09-24 12:28:27 +00:00
- Reorganized functions for clarity in `` BaseClass `` (save/load functions close to each other, private
2020-10-11 21:22:12 +00:00
functions at top)
2020-09-24 12:28:27 +00:00
- Clarified docstrings on what is saved and loaded to/from files
- Simplified `` save_to_zip_file `` function by removing duplicate code
- Store library version along with the saved models
2020-10-04 15:12:35 +00:00
- DQN loss is now logged
2020-08-05 10:12:02 +00:00
Documentation:
^^^^^^^^^^^^^^
2020-08-28 09:36:33 +00:00
- Added `` StopTrainingOnMaxEpisodes `` details and example (@xicocaio)
2020-09-29 17:41:14 +00:00
- Updated custom policy section (added custom feature extractor example)
- Re-enable `` sphinx_autodoc_typehints ``
2020-10-02 17:05:55 +00:00
- Updated doc style for type hints and remove duplicated type hints
2020-08-28 09:36:33 +00:00
2020-08-05 10:12:02 +00:00
2020-08-03 20:38:54 +00:00
Pre-Release 0.8.0 (2020-08-03)
2020-06-17 10:47:09 +00:00
------------------------------
2020-08-03 20:38:54 +00:00
**DQN, DDPG, bug fixes and performance matching for Atari games**
2020-06-17 10:47:09 +00:00
Breaking Changes:
^^^^^^^^^^^^^^^^^
2020-06-29 09:16:54 +00:00
- `` AtariWrapper `` and other Atari wrappers were updated to match SB2 ones
2020-06-17 12:00:49 +00:00
- `` save_replay_buffer `` now receives as argument the file path instead of the folder path (@tirafesi)
2020-07-06 22:02:51 +00:00
- Refactored `` Critic `` class for `` TD3 `` and `` SAC `` , it is now called `` ContinuousCritic ``
and has an additional parameter `` n_critics ``
2020-07-16 12:14:22 +00:00
- `` SAC `` and `` TD3 `` now accept an arbitrary number of critics (e.g. `` policy_kwargs=dict(n_critics=3) `` )
2020-10-11 21:22:12 +00:00
instead of only 2 previously
2020-06-17 10:47:09 +00:00
New Features:
^^^^^^^^^^^^^
2020-06-29 09:16:54 +00:00
- Added `` DQN `` Algorithm (@Artemis-Skade)
- Buffer dtype is now set according to action and observation spaces for `` ReplayBuffer ``
- Added warning when allocation of a buffer may exceed the available memory of the system
when `` psutil `` is available
2020-07-02 22:14:21 +00:00
- Saving models now automatically creates the necessary folders and raises appropriate warnings (@PartiallyTyped)
- Refactored opening paths for saving and loading to use strings, pathlib or io.BufferedIOBase (@PartiallyTyped)
2020-07-16 12:14:22 +00:00
- Added `` DDPG `` algorithm as a special case of `` TD3 `` .
2020-07-08 02:03:46 +00:00
- Introduced `` BaseModel `` abstract parent for `` BasePolicy `` , which critics inherit from.
2020-06-17 10:47:09 +00:00
Bug Fixes:
^^^^^^^^^^
2020-06-20 16:01:37 +00:00
- Fixed a bug in the `` close() `` method of `` SubprocVecEnv `` , causing wrappers further down in the wrapper stack to not be closed. (@NeoExtended)
2020-06-29 15:58:55 +00:00
- Fix target for updating q values in SAC: the entropy term was not conditioned by terminals states
2020-07-21 08:12:39 +00:00
- Use `` cloudpickle.load `` instead of `` pickle.load `` in `` CloudpickleWrapper `` . (@shwang)
2020-07-25 20:35:48 +00:00
- Fixed a bug with orthogonal initialization when `bias=False`
2020-07-29 19:19:41 +00:00
- Fixed approximate entropy calculation in PPO and A2C. (@andyshih12)
2020-07-30 18:48:30 +00:00
- Fixed DQN target network sharing feature extractor with the main network.
2020-08-03 20:22:51 +00:00
- Fixed storing correct `` dones `` in on-policy algorithm rollout collection. (@andyshih12)
- Fixed number of filters in final convolutional layer in NatureCNN to match original implementation.
2020-06-17 10:47:09 +00:00
Deprecations:
^^^^^^^^^^^^^
Others:
^^^^^^^
2020-06-29 09:16:54 +00:00
- Refactored off-policy algorithm to share the same `` .learn() `` method
- Split the `` collect_rollout() `` method for off-policy algorithms
- Added `` _on_step() `` for off-policy base class
- Optimized replay buffer size by removing the need of `` next_observations `` numpy array
2020-07-17 13:53:28 +00:00
- Optimized polyak updates (1.5-1.95 speedup) through inplace operations (@PartiallyTyped)
2020-07-16 14:12:16 +00:00
- Switch to `` black `` codestyle and added `` make format `` , `` make check-codestyle `` and `` commit-checks ``
2020-07-16 09:54:37 +00:00
- Ignored errors from newer pytype version
2020-07-16 12:14:22 +00:00
- Added a check when using `` gSDE ``
2020-07-16 14:12:16 +00:00
- Removed codacy dependency from Dockerfile
2020-08-03 20:22:51 +00:00
- Added `` common.sb2_compat.RMSpropTFLike `` optimizer, which corresponds closer to the implementation of RMSprop from Tensorflow.
2020-06-17 10:47:09 +00:00
Documentation:
^^^^^^^^^^^^^^
- Updated notebook links
2020-06-25 10:14:56 +00:00
- Fixed a typo in the section of Enjoy a Trained Agent, in RL Baselines3 Zoo README. (@blurLake)
2020-06-30 13:03:02 +00:00
- Added Unity reacher to the projects page (@koulakis)
2020-07-06 22:02:51 +00:00
- Added PyBullet colab notebook
2020-07-10 08:38:35 +00:00
- Fixed typo in PPO example code (@joeljosephjin)
2020-07-31 09:10:48 +00:00
- Fixed typo in custom policy doc (@RaphaelWag)
2020-06-17 10:47:09 +00:00
2020-06-04 09:22:12 +00:00
2020-06-10 16:58:35 +00:00
Pre-Release 0.7.0 (2020-06-10)
2020-06-04 09:22:12 +00:00
------------------------------
2020-06-10 16:58:35 +00:00
**Hotfix for PPO/A2C + gSDE, internal refactoring and bug fixes**
2020-06-04 09:22:12 +00:00
Breaking Changes:
^^^^^^^^^^^^^^^^^
- `` render() `` method of `` VecEnvs `` now only accept one argument: `` mode ``
2020-06-09 11:54:18 +00:00
- Created new file common/torch_layers.py, similar to SB refactoring
2020-06-10 16:58:35 +00:00
2020-06-09 11:54:18 +00:00
- Contains all PyTorch network layer definitions and feature extractors: `` MlpExtractor `` , `` create_mlp `` , `` NatureCNN ``
2020-06-04 09:22:12 +00:00
2020-06-09 11:54:18 +00:00
- Renamed `` BaseRLModel `` to `` BaseAlgorithm `` (along with offpolicy and onpolicy variants)
- Moved on-policy and off-policy base algorithms to `` common/on_policy_algorithm.py `` and `` common/off_policy_algorithm.py `` , respectively.
2020-06-10 16:58:35 +00:00
- Moved `` PPOPolicy `` to `` ActorCriticPolicy `` in common/policies.py
2020-06-09 11:54:18 +00:00
- Moved `` PPO `` (algorithm class) into `` OnPolicyAlgorithm `` (`` common/on_policy_algorithm.py `` ), to be shared with A2C
2020-06-10 16:58:35 +00:00
- Moved following functions from `` BaseAlgorithm `` :
2020-06-09 11:54:18 +00:00
- `` _load_from_file `` to `` load_from_zip_file `` (save_util.py)
- `` _save_to_file_zip `` to `` save_to_zip_file `` (save_util.py)
- `` safe_mean `` to `` safe_mean `` (utils.py)
- `` check_env `` to `` check_for_correct_spaces `` (utils.py. Renamed to avoid confusion with environment checker tools)
- Moved static function `` _is_vectorized_observation `` from common/policies.py to common/utils.py under name `` is_vectorized_observation `` .
2020-06-10 10:09:04 +00:00
- Removed `` {save,load}_running_average `` functions of `` VecNormalize `` in favor of `` load/save `` .
- Removed `` use_gae `` parameter from `` RolloutBuffer.compute_returns_and_advantage `` .
2020-06-10 16:58:35 +00:00
2020-06-04 09:22:12 +00:00
New Features:
^^^^^^^^^^^^^
Bug Fixes:
^^^^^^^^^^
- Fixed `` render() `` method for `` VecEnvs ``
2020-06-09 11:54:18 +00:00
- Fixed `` seed() `` method for `` SubprocVecEnv ``
2020-06-08 09:15:10 +00:00
- Fixed loading on GPU for testing when using gSDE and `` deterministic=False ``
2020-06-09 11:54:18 +00:00
- Fixed `` register_policy `` to allow re-registering same policy for same sub-class (i.e. assign same value to same key).
2020-06-10 16:58:35 +00:00
- Fixed a bug where the gradient was passed when using `` gSDE `` with `` PPO `` /`` A2C `` , this does not affect `` SAC ``
2020-06-04 09:22:12 +00:00
Deprecations:
^^^^^^^^^^^^^
Others:
^^^^^^^
- Re-enable unsafe `` fork `` start method in the tests (was causing a deadlock with tensorflow)
2020-06-09 11:54:18 +00:00
- Added a test for seeding `` SubprocVecEnv `` and rendering
- Fixed reference in NatureCNN (pointed to older version with different network architecture)
- Fixed comments saying "CxWxH" instead of "CxHxW" (same style as in torch docs / commonly used)
- Added bit further comments on register/getting policies ("MlpPolicy", "CnnPolicy").
- Renamed `` progress `` (value from 1 in start of training to 0 in end) to `` progress_remaining `` .
- Added `` policies.py `` files for A2C/PPO, which define MlpPolicy/CnnPolicy (renamed ActorCriticPolicies).
2020-06-10 10:09:04 +00:00
- Added some missing tests for `` VecNormalize `` , `` VecCheckNan `` and `` PPO `` .
2020-06-04 09:22:12 +00:00
Documentation:
^^^^^^^^^^^^^^
2020-06-09 11:54:18 +00:00
- Added a paragraph on "MlpPolicy"/"CnnPolicy" and policy naming scheme under "Developer Guide"
- Fixed second-level listing in changelog
2020-06-04 09:22:12 +00:00
2020-06-17 10:47:09 +00:00
2020-06-01 11:09:47 +00:00
Pre-Release 0.6.0 (2020-06-01)
2020-05-05 14:28:38 +00:00
------------------------------
2020-06-01 11:09:47 +00:00
**Tensorboard support, refactored logger**
2020-05-05 14:28:38 +00:00
Breaking Changes:
^^^^^^^^^^^^^^^^^
2020-05-08 13:00:34 +00:00
- Remove State-Dependent Exploration (SDE) support for `` TD3 ``
2020-06-01 09:55:44 +00:00
- Methods were renamed in the logger:
2020-06-10 16:58:35 +00:00
2020-06-01 09:55:44 +00:00
- `` logkv `` -> `` record `` , `` writekvs `` -> `` write `` , `` writeseq `` -> `` write_sequence `` ,
- `` logkvs `` -> `` record_dict `` , `` dumpkvs `` -> `` dump `` ,
- `` getkvs `` -> `` get_log_dict `` , `` logkv_mean `` -> `` record_mean `` ,
2020-05-05 14:28:38 +00:00
New Features:
^^^^^^^^^^^^^
- Added env checker (Sync with Stable Baselines)
- Added `` VecCheckNan `` and `` VecVideoRecorder `` (Sync with Stable Baselines)
2020-05-05 14:54:33 +00:00
- Added determinism tests
2020-05-19 08:19:16 +00:00
- Added `` cmd_util `` and `` atari_wrappers ``
2020-05-18 12:42:13 +00:00
- Added support for `` MultiDiscrete `` and `` MultiBinary `` observation spaces (@rolandgvc)
- Added `` MultiCategorical `` and `` Bernoulli `` distributions for PPO/A2C (@rolandgvc)
2020-06-01 09:55:44 +00:00
- Added support for logging to tensorboard (@rolandgvc)
2020-05-27 07:53:01 +00:00
- Added `` VectorizedActionNoise `` for continuous vectorized environments (@PartiallyTyped)
2020-06-01 09:55:44 +00:00
- Log evaluation in the `` EvalCallback `` using the logger
2020-05-05 14:28:38 +00:00
Bug Fixes:
^^^^^^^^^^
2020-05-05 15:21:56 +00:00
- Fixed a bug that prevented model trained on cpu to be loaded on gpu
2020-05-07 14:08:23 +00:00
- Fixed version number that had a new line included
2020-05-09 21:10:49 +00:00
- Fixed weird seg fault in docker image due to FakeImageEnv by reducing screen size
2020-05-24 14:44:44 +00:00
- Fixed `` sde_sample_freq `` that was not taken into account for SAC
2020-06-01 09:55:44 +00:00
- Pass logger module to `` BaseCallback `` otherwise they cannot write in the one used by the algorithms
2020-05-05 14:28:38 +00:00
Deprecations:
^^^^^^^^^^^^^
Others:
^^^^^^^
- Renamed to Stable-Baseline3
2020-05-07 14:08:23 +00:00
- Added Dockerfile
- Sync `` VecEnvs `` with Stable-Baselines
2020-05-08 10:43:42 +00:00
- Update requirement: `` gym>=0.17 ``
2020-05-11 10:28:43 +00:00
- Added `` .readthedoc.yml `` file
2020-05-12 15:55:01 +00:00
- Added `` flake8 `` and `` make lint `` command
- Added Github workflow
2020-06-01 11:09:47 +00:00
- Added warning when passing both `` train_freq `` and `` n_episodes_rollout `` to Off-Policy Algorithms
2020-05-05 14:28:38 +00:00
Documentation:
^^^^^^^^^^^^^^
2020-05-07 08:10:51 +00:00
- Added most documentation (adapted from Stable-Baselines)
2020-05-09 11:01:15 +00:00
- Added link to CONTRIBUTING.md in the README (@kinalmehta)
2020-05-15 11:54:06 +00:00
- Added gSDE project and update docstrings accordingly
2020-06-01 08:37:42 +00:00
- Fix `` TD3 `` example code block
2020-05-05 14:28:38 +00:00
2020-05-05 12:49:32 +00:00
Pre-Release 0.5.0 (2020-05-05)
2020-03-20 09:09:09 +00:00
------------------------------
2020-05-05 12:49:32 +00:00
**CnnPolicy support for image observations, complete saving/loading for policies**
2020-03-20 09:09:09 +00:00
Breaking Changes:
^^^^^^^^^^^^^^^^^
2020-04-20 13:59:44 +00:00
- Previous loading of policy weights is broken and replace by the new saving/loading for policy
2020-03-20 09:09:09 +00:00
New Features:
^^^^^^^^^^^^^
2020-04-22 11:14:22 +00:00
- Added `` optimizer_class `` and `` optimizer_kwargs `` to `` policy_kwargs `` in order to easily
2020-04-17 13:13:45 +00:00
customizer optimizers
2020-04-20 13:59:44 +00:00
- Complete independent save/load for policies
2020-04-22 11:14:22 +00:00
- Add `` CnnPolicy `` and `` VecTransposeImage `` to support images as input
2020-04-20 13:59:44 +00:00
2020-03-20 09:09:09 +00:00
Bug Fixes:
^^^^^^^^^^
2020-04-17 13:13:45 +00:00
- Fixed `` reset_num_timesteps `` behavior, so `` env.reset() `` is not called if `` reset_num_timesteps=True ``
2020-04-20 13:43:11 +00:00
- Fixed `` squashed_output `` that was not pass to policy constructor for `` SAC `` and `` TD3 `` (would result in scaled actions for unscaled action spaces)
2020-03-20 09:09:09 +00:00
Deprecations:
^^^^^^^^^^^^^
Others:
^^^^^^^
2020-04-17 13:13:45 +00:00
- Cleanup rollout return
2020-04-20 13:43:11 +00:00
- Added `` get_device `` util to manage PyTorch devices
2020-05-05 12:49:32 +00:00
- Added type hints to logger + use f-strings
2020-03-20 09:09:09 +00:00
Documentation:
^^^^^^^^^^^^^^
2020-04-17 13:13:45 +00:00
Pre-Release 0.4.0 (2020-02-14)
2020-04-17 10:36:27 +00:00
------------------------------
2020-04-17 13:13:45 +00:00
**Proper pre-processing, independent save/load for policies**
2020-04-17 10:36:27 +00:00
Breaking Changes:
^^^^^^^^^^^^^^^^^
2020-04-17 13:13:45 +00:00
- Removed CEMRL
- Model saved with previous versions cannot be loaded (because of the pre-preprocessing)
2020-04-17 10:36:27 +00:00
New Features:
^^^^^^^^^^^^^
2020-04-17 13:13:45 +00:00
- Add support for `` Discrete `` observation spaces
- Add saving/loading for policy weights, so the policy can be used without the model
2020-04-17 10:36:27 +00:00
Bug Fixes:
^^^^^^^^^^
2020-04-17 13:13:45 +00:00
- Fix type hint for activation functions
Deprecations:
^^^^^^^^^^^^^
2020-04-17 10:36:27 +00:00
Others:
^^^^^^^
2020-04-17 13:13:45 +00:00
- Refactor handling of observation and action spaces
- Refactored features extraction to have proper preprocessing
- Refactored action distributions
2020-04-17 10:36:27 +00:00
2020-02-14 13:39:24 +00:00
2020-03-19 10:11:36 +00:00
Pre-Release 0.3.0 (2020-02-14)
2020-01-20 15:19:35 +00:00
------------------------------
2019-09-26 09:46:40 +00:00
2020-03-19 10:11:36 +00:00
**Bug fixes, sync with Stable-Baselines, code cleanup**
2020-02-14 13:39:24 +00:00
Breaking Changes:
^^^^^^^^^^^^^^^^^
2020-03-10 16:43:54 +00:00
- Removed default seed
- Bump dependencies (PyTorch and Gym)
2020-03-18 14:11:19 +00:00
- `` predict() `` now returns a tuple to match Stable-Baselines behavior
2020-02-14 13:39:24 +00:00
New Features:
^^^^^^^^^^^^^
2020-03-13 10:43:12 +00:00
- Better logging for `` SAC `` and `` PPO ``
2020-02-14 13:39:24 +00:00
Bug Fixes:
^^^^^^^^^^
2020-03-12 11:34:25 +00:00
- Synced callbacks with Stable-Baselines
2020-03-19 10:11:36 +00:00
- Fixed colors in `` results_plotter ``
2020-03-19 09:19:48 +00:00
- Fix entropy computation (now summed over action dim)
2020-02-14 13:39:24 +00:00
Others:
^^^^^^^
2020-02-21 13:50:28 +00:00
- SAC with SDE now sample only one matrix
2020-03-09 18:05:22 +00:00
- Added `` clip_mean `` parameter to SAC policy
2020-03-10 15:55:13 +00:00
- Buffers now return `` NamedTuple ``
2020-03-10 16:43:54 +00:00
- More typing
2020-03-11 15:35:13 +00:00
- Add test for `` expln ``
2020-03-16 13:01:32 +00:00
- Renamed `` learning_rate `` to `` lr_schedule ``
2020-03-18 14:11:19 +00:00
- Add `` version.txt ``
2020-03-19 09:19:48 +00:00
- Add more tests for distribution
2020-02-14 13:39:24 +00:00
Documentation:
^^^^^^^^^^^^^^
2020-03-16 13:01:32 +00:00
- Deactivated `` sphinx_autodoc_typehints `` extension
2020-02-14 13:39:24 +00:00
Pre-Release 0.2.0 (2020-02-14)
------------------------------
**Python 3.6+ required, type checking, callbacks, doc build**
2019-09-26 09:46:40 +00:00
Breaking Changes:
^^^^^^^^^^^^^^^^^
2020-05-05 13:02:35 +00:00
- Python 2 support was dropped, Stable Baselines3 now requires Python 3.6 or above
2020-03-09 18:05:22 +00:00
- Return type of `` evaluation.evaluate_policy() `` has been changed
2020-02-03 14:50:40 +00:00
- Refactored the replay buffer to avoid transformation between PyTorch and NumPy
2020-02-05 12:10:02 +00:00
- Created `OffPolicyRLModel`
2020-02-14 12:42:16 +00:00
- Remove deprecated JSON format for `Monitor`
2019-09-26 09:46:40 +00:00
New Features:
^^^^^^^^^^^^^
2020-03-09 18:05:22 +00:00
- Add `` seed() `` method to `` VecEnv `` class
2020-01-27 13:32:31 +00:00
- Add support for Callback (cf https://github.com/hill-a/stable-baselines/pull/644)
2020-02-05 12:10:02 +00:00
- Add methods for saving and loading replay buffer
2020-03-09 18:05:22 +00:00
- Add `` extend() `` method to the buffers
- Add `` get_vec_normalize_env() `` to `` BaseRLModel `` to retrieve `` VecNormalize `` wrapper when it exists
- Add `` results_plotter `` from Stable Baselines
- Improve `` predict() `` method to handle different type of observations (single, vectorized, ...)
2019-09-26 09:46:40 +00:00
Bug Fixes:
^^^^^^^^^^
2020-01-31 12:06:55 +00:00
- Fix loading model on CPU that were trained on GPU
2020-03-09 18:05:22 +00:00
- Fix `` reset_num_timesteps `` that was not used
2020-02-11 16:22:03 +00:00
- Fix entropy computation for squashed Gaussian (approximate it now)
- Fix seeding when using multiple environments (different seed per env)
2019-09-26 09:46:40 +00:00
2020-01-20 15:19:35 +00:00
Others:
^^^^^^^
2020-01-22 16:23:42 +00:00
- Add type check
- Converted all format string to f-strings
2020-03-09 18:05:22 +00:00
- Add test for `` OrnsteinUhlenbeckActionNoise ``
- Add type aliases in `` common.type_aliases ``
2020-01-20 15:19:35 +00:00
Documentation:
^^^^^^^^^^^^^^
- fix documentation build
Pre-Release 0.1.0 (2020-01-20)
------------------------------
**First Release: base algorithms and state-dependent exploration**
New Features:
^^^^^^^^^^^^^
2020-03-09 18:05:22 +00:00
- Initial release of A2C, CEM-RL, PPO, SAC and TD3, working only with `` Box `` input space
2020-01-20 15:19:35 +00:00
- State-Dependent Exploration (SDE) for A2C, PPO, SAC and TD3
2019-09-26 09:46:40 +00:00
Maintainers
-----------
2020-05-05 14:28:38 +00:00
Stable-Baselines3 is currently maintained by `Antonin Raffin`_ (aka `@araffin`_ ), `Ashley Hill`_ (aka @hill-a),
`Maximilian Ernestus`_ (aka @erniejunior), `Adam Gleave`_ (`@AdamGleave`_ ) and `Anssi Kanervisto`_ (aka `@Miffyli`_ ).
2019-09-26 09:46:40 +00:00
2020-05-05 14:28:38 +00:00
.. _Ashley Hill: https://github.com/hill-a
2019-09-26 09:46:40 +00:00
.. _Antonin Raffin: https://araffin.github.io/
2020-05-05 14:28:38 +00:00
.. _Maximilian Ernestus: https://github.com/erniejunior
.. _Adam Gleave: https://gleave.me/
2019-09-26 09:46:40 +00:00
.. _@araffin: https://github.com/araffin
2020-05-05 14:28:38 +00:00
.. _@AdamGleave: https://github.com/adamgleave
.. _Anssi Kanervisto: https://github.com/Miffyli
.. _@Miffyli: https://github.com/Miffyli
2019-09-26 09:46:40 +00:00
Contributors:
-------------
In random order...
2020-03-09 18:05:22 +00:00
Thanks to the maintainers of V2: @hill-a @enerijunior @AdamGleave @Miffyli
And all the contributors:
@bjmuld @iambenzo @iandanforth @r7vme @brendenpetersen @huvar @abhiskk @JohannesAck
@EliasHasle @mrakgr @Bleyddyn @antoine-galataud @junhyeokahn @AdamGleave @keshaviyengar @tperol
@XMaster96 @kantneel @Pastafarianist @GerardMaggiolino @PatrickWalter214 @yutingsz @sc420 @Aaahh @billtubbs
@Miffyli @dwiel @miguelrass @qxcv @jaberkow @eavelardev @ruifeng96150 @pedrohbtp @srivatsankrishnan @evilsocket
2020-11-02 10:45:08 +00:00
@MarvineGothic @jdossgollin @stheid @SyllogismRXS @rusu24edward @jbulow @Antymon @seheevic @justinkterry @edbeeching
2020-06-01 08:37:42 +00:00
@flodorner @KuKuXia @NeoExtended @PartiallyTyped @mmcenta @richardwu @kinalmehta @rolandgvc @tkelestemur @mloo3
2020-08-28 09:36:33 +00:00
@tirafesi @blurLake @koulakis @joeljosephjin @shwang @rk37 @andyshih12 @RaphaelWag @xicocaio
2020-10-22 09:56:43 +00:00
@diditforlulz273 @liorcohen5 @ManifoldFR @mloo3 @SwamyDev @wmmc88 @megan-klaiber
