Megan Klaiber dd6e361204 Implement HER (#120) ... * Added working her version, Online sampling is missing. * Updated test_her. * Added first version of online her sampling. Still problems with tensor dimensions. * Reformat * Fixed tests * Added some comments. * Updated changelog. * Add missing init file * Fixed some small bugs. * Reduced arguments for HER, small changes. * Added getattr. Fixed bug for online sampling. * Updated save/load funtions. Small changes. * Added her to init. * Updated save method. * Updated her ratio. * Move obs_wrapper * Added DQN test. * Fix potential bug * Offline and online her share same sample_goal function. * Changed lists into arrays. * Updated her test. * Fix online sampling * Fixed action bug. Updated time limit for episodes. * Updated convert_dict method to take keys as arguments. * Renamed obs dict wrapper. * Seed bit flipping env * Remove get_episode_dict * Add fast online sampling version * Added documentation. * Vectorized reward computation * Vectorized goal sampling * Update time limit for episodes in online her sampling. * Fix max episode length inference * Bug fix for Fetch envs * Fix for HER + gSDE * Reformat (new black version) * Added info dict to compute new reward. Check her_replay_buffer again. * Fix info buffer * Updated done flag. * Fixes for gSDE * Offline her version uses now HerReplayBuffer as episode storage. * Fix num_timesteps computation * Fix get torch params * Vectorized version for offline sampling. * Modified offline her sampling to use sample method of her_replay_buffer * Updated HER tests. * Updated documentation * Cleanup docstrings * Updated to review comments * Fix pytype * Update according to review comments. * Removed random goal strategy. Updated sample transitions. * Updated migration. Removed time signal removal. * Update doc * Fix potential load issue * Add VecNormalize support for dict obs * Updated saving/loading replay buffer for HER. * Fix test memory usage * Fixed save/load replay buffer. * Fixed save/load replay buffer * Fixed transition index after loading replay buffer in online sampling * Better error handling * Add tests for get_time_limit * More tests for VecNormalize with dict obs * Update doc * Improve HER description * Add test for sde support * Add comments * Add comments * Remove check that was always valid * Fix for terminal observation * Updated buffer size in offline version and reset of HER buffer * Reformat * Update doc * Remove np.empty + add doc * Fix loading * Updated loading replay buffer * Separate online and offline sampling + bug fixes * Update tensorboard log name * Version bump * Bug fix for special case Co-authored-by: Antonin Raffin <antonin.raffin@dlr.de> Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>		2020-10-22 11:56:43 +02:00
..
algos.rst	Implement DDPG (#92)	2020-07-16 14:14:22 +02:00
callbacks.rst	Add StopTrainingOnMaxEpisodes to callback collection (#147)	2020-08-28 11:36:33 +02:00
checking_nan.rst	Add base doc	2020-05-07 10:10:51 +02:00
custom_env.rst	Add base doc	2020-05-07 10:10:51 +02:00
custom_policy.rst	Add custom arch for off-policy actor/critic networks (#182)	2020-10-13 12:01:33 +02:00
developer.rst	Review of code (A2C, PPO and refactoring) (#35)	2020-06-09 13:54:18 +02:00
examples.rst	Implement HER (#120)	2020-10-22 11:56:43 +02:00
install.rst	Migration Guide (#123)	2020-10-11 23:22:12 +02:00
migration.rst	Implement HER (#120)	2020-10-22 11:56:43 +02:00
quickstart.rst	[ci skip] Simplify quickstart example	2020-05-11 15:32:01 +02:00
rl.rst	Add base doc	2020-05-07 10:10:51 +02:00
rl_tips.rst	Update doc	2020-05-19 10:40:52 +02:00
rl_zoo.rst	Update rl_zoo.rst (#72)	2020-06-25 12:14:56 +02:00
tensorboard.rst	Add support to log videos via tensorboard (#196)	2020-10-22 11:33:58 +02:00
vec_envs.rst	Add base doc	2020-05-07 10:10:51 +02:00