Adds info on split tensorboard graphs (#989)

* Add info on split tensorboard graphs.

* Change wording to make it look better.

* Update changelog.rst

* Rephrase and add link to issue

Co-authored-by: Antonin Raffin <antonin.raffin@ensta.org>

docs/guide/tensorboard.rst

@@ -26,10 +26,20 @@ You can also define custom logging name when training (by default it is the algo
     model.learn(total_timesteps=10_000, tb_log_name="first_run")
     # Pass reset_num_timesteps=False to continue the training curve in tensorboard
     # By default, it will create a new curve
+    # Keep tb_log_name constant to have continuous curve (see note below)
     model.learn(total_timesteps=10_000, tb_log_name="second_run", reset_num_timesteps=False)
     model.learn(total_timesteps=10_000, tb_log_name="third_run", reset_num_timesteps=False)
 
 
+.. note::
+    If you specify different ``tb_log_name`` in subsequent runs, you will have split graphs, like in the figure below.
+    If you want them to be continuous, you must keep the same ``tb_log_name`` (see `issue #975 <https://github.com/DLR-RM/stable-baselines3/issues/975#issuecomment-1198992211>`_).
+    And, if you still managed to get your graphs split by other means, just put tensorboard log files into the same folder.
+
+    .. image:: ../_static/img/split_graph.png
+      :width: 330
+      :alt: split_graph
+
 Once the learn function is called, you can monitor the RL agent during or after the training, with the following bash command:
 
 .. code-block:: bash

docs/misc/changelog.rst

@@ -29,6 +29,7 @@ Others:
 Documentation:
 ^^^^^^^^^^^^^^
 - Fix typo in docstring "nature" -> "Nature" (@Melanol)
+- Add info on split tensorboard logs into (@Melanol)
 
 
 Release 1.6.0 (2022-07-11)