Merge branch 'master' into fix/close-method-memory-leak

Author: RPalmr

.github/workflows/ci.yml

@@ -20,17 +20,17 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ["3.9", "3.10", "3.11", "3.12"]
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
         include:
           # Default version
           - gymnasium-version: "1.0.0"
           # Add a new config to test gym<1.0
           - python-version: "3.10"
             gymnasium-version: "0.29.1"
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v6
       - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v6
         with:
           python-version: ${{ matrix.python-version }}
       - name: Install dependencies
@@ -40,7 +40,8 @@ jobs:
           pip install uv
           # cpu version of pytorch
           # See https://github.com/astral-sh/uv/issues/1497
-          uv pip install --system torch==2.3.1+cpu --index https://download.pytorch.org/whl/cpu
+          # Need Pytorch 2.9+ for Python 3.13
+          uv pip install --system torch==2.9.1+cpu --index https://download.pytorch.org/whl/cpu
 
           uv pip install --system .[extra,tests,docs]
           # Use headless version

README.md

@@ -103,7 +103,7 @@ It provides a minimal number of features compared to SB3 but can be much faster
 **Note:** Stable-Baselines3 supports PyTorch >= 2.3
 
 ### Prerequisites
-Stable Baselines3 requires Python 3.9+.
+Stable Baselines3 requires Python 3.10+.
 
 #### Windows
 

docs/guide/install.rst

@@ -7,7 +7,7 @@ Installation
 Prerequisites
 -------------
 
-Stable-Baselines3 requires python 3.9+ and PyTorch >= 2.3
+Stable-Baselines3 requires python 3.10+ and PyTorch >= 2.3
 
 Windows
 ~~~~~~~

docs/misc/changelog.rst

@@ -3,23 +3,22 @@
 Changelog
 ==========
 
-Release 2.7.1a3 (WIP)
+
+Release 2.8.0a2 (WIP)
 --------------------------
 
 Breaking Changes:
 ^^^^^^^^^^^^^^^^^
+- Removed support for Python 3.9, please upgrade to Python >= 3.10
+- Set ``strict=True`` for every call to ``zip(...)``
 
 New Features:
 ^^^^^^^^^^^^^
-- ``RolloutBuffer`` and ``DictRolloutBuffer`` now uses the actual observation / action space ``dtype`` (instead of float32), this should save memory (@Trenza1ore)
+- Added official support for Python 3.13
 
 Bug Fixes:
 ^^^^^^^^^^
-- Fixed env checker to properly handle ``Sequence`` observation spaces when nested inside composite spaces (``Dict``, ``Tuple``, ``OneOf``) (@copilot)
-- Update env checker to warn users when using Graph space (@dhruvmalik007).
-- Fixed memory leak in ``VecVideoRecorder`` where ``recorded_frames`` stayed in memory due to reference in the moviepy clip (@copilot)
-- Remove double space in `StopTrainingOnRewardThreshold` callback message (@sea-bass)
-- Add close method to BaseAlgorithm to prevent memory leaks in sequential training loops (#1966)
+- Fixed saving and loading of Torch compiled models (using ``th.compile()``) by updating ``get_parameters()``
 
 `SB3-Contrib`_
 ^^^^^^^^^^^^^^
@@ -32,9 +31,51 @@ Bug Fixes:
 
 Deprecations:
 ^^^^^^^^^^^^^
+- ``zip_strict()`` is not needed anymore since Python 3.10, please use ``zip(..., strict=True)`` instead
 
 Others:
 ^^^^^^^
+- Updated to Python 3.10+ annotations
+- Removed some unused variables (@unexploredtest)
+- Improved type hints for distributions
+- Simplified zip file loading by removing Python 3.6 workaround and enabling ``weights_only=True`` (PyTorch 2.x)
+- Sped up saving/loading tests
+
+Documentation:
+^^^^^^^^^^^^^^
+
+
+Release 2.7.1 (2025-12-05)
+--------------------------
+
+.. warning::
+
+    Stable-Baselines3 (SB3) v2.7.1 will be the last one supporting Python 3.9 (end of life in October 2025).
+    We highly recommended you to upgrade to Python >= 3.10.
+
+
+Breaking Changes:
+^^^^^^^^^^^^^^^^^
+
+New Features:
+^^^^^^^^^^^^^
+- ``RolloutBuffer`` and ``DictRolloutBuffer`` now uses the actual observation / action space ``dtype`` (instead of float32), this should save memory (@Trenza1ore)
+
+Bug Fixes:
+^^^^^^^^^^
+- Fixed env checker to properly handle ``Sequence`` observation spaces when nested inside composite spaces (``Dict``, ``Tuple``, ``OneOf``) (@copilot)
+- Update env checker to warn users when using Graph space (@dhruvmalik007).
+- Fixed memory leak in ``VecVideoRecorder`` where ``recorded_frames`` stayed in memory due to reference in the moviepy clip (@copilot)
+- Remove double space in `StopTrainingOnRewardThreshold` callback message (@sea-bass)
+- Add close method to BaseAlgorithm to prevent memory leaks in sequential training loops (#1966)
+
+`SB3-Contrib`_
+^^^^^^^^^^^^^^
+- Fixed tensorboard log name for ``MaskablePPO``
+
+`SBX`_ (SB3 + Jax)
+^^^^^^^^^^^^^^^^^^
+- Added ``CnnPolicy`` to PPO
 
 Documentation:
 ^^^^^^^^^^^^^^
@@ -47,7 +88,7 @@ Documentation:
 - Updated link to paper of community project DeepNetSlice (@AlexPasqua)
 - Added example usage of Tensorflow JS
 - Included exact versions in ONNX JS and example project
-- Made step 2 (`pip install`) of `CONTRIBUTING.md` more robust 
+- Made step 2 (`pip install`) of `CONTRIBUTING.md` more robust
 
 
 Release 2.7.0 (2025-07-25)
@@ -1904,4 +1945,4 @@ And all the contributors:
 @DavyMorgan @luizapozzobon @Bonifatius94 @theSquaredError @harveybellini @DavyMorgan @FieteO @jonasreiher @npit @WeberSamuel @troiganto
 @lutogniew @lbergmann1 @lukashass @BertrandDecoster @pseudo-rnd-thoughts @stefanbschneider @kyle-he @PatrickHelm @corentinlger
 @marekm4 @stagoverflow @rushitnshah @markscsmith @NickLucche @cschindlbeck @peteole @jak3122 @will-maclean
-@brn-dev @jmacglashan @kplers @MarcDcls @chrisgao99 @pstahlhofen @akanto @Trenza1ore @JonathanColetti
+@brn-dev @jmacglashan @kplers @MarcDcls @chrisgao99 @pstahlhofen @akanto @Trenza1ore @JonathanColetti @unexploredtest

pyproject.toml

@@ -1,8 +1,8 @@
 [tool.ruff]
 # Same as Black.
 line-length = 127
-# Assume Python 3.9
-target-version = "py39"
+# Assume Python 3.10
+target-version = "py310"
 
 [tool.ruff.lint]
 # See https://beta.ruff.rs/docs/rules/

setup.py

@@ -135,7 +135,7 @@
     long_description=long_description,
     long_description_content_type="text/markdown",
     version=__version__,
-    python_requires=">=3.9",
+    python_requires=">=3.10",
     # PyPI package information.
     project_urls={
         "Code": "https://github.com/DLR-RM/stable-baselines3",
@@ -147,10 +147,10 @@
     },
     classifiers=[
         "Programming Language :: Python :: 3",
-        "Programming Language :: Python :: 3.9",
         "Programming Language :: Python :: 3.10",
         "Programming Language :: Python :: 3.11",
         "Programming Language :: Python :: 3.12",
+        "Programming Language :: Python :: 3.13",
     ],
 )
 

stable_baselines3/a2c/a2c.py

@@ -1,4 +1,4 @@
-from typing import Any, ClassVar, Optional, TypeVar, Union
+from typing import Any, ClassVar, TypeVar
 
 import torch as th
 from gymnasium import spaces
@@ -65,9 +65,9 @@ class A2C(OnPolicyAlgorithm):
 
     def __init__(
         self,
-        policy: Union[str, type[ActorCriticPolicy]],
-        env: Union[GymEnv, str],
-        learning_rate: Union[float, Schedule] = 7e-4,
+        policy: str | type[ActorCriticPolicy],
+        env: GymEnv | str,
+        learning_rate: float | Schedule = 7e-4,
         n_steps: int = 5,
         gamma: float = 0.99,
         gae_lambda: float = 1.0,
@@ -78,15 +78,15 @@ def __init__(
         use_rms_prop: bool = True,
         use_sde: bool = False,
         sde_sample_freq: int = -1,
-        rollout_buffer_class: Optional[type[RolloutBuffer]] = None,
-        rollout_buffer_kwargs: Optional[dict[str, Any]] = None,
+        rollout_buffer_class: type[RolloutBuffer] | None = None,
+        rollout_buffer_kwargs: dict[str, Any] | None = None,
         normalize_advantage: bool = False,
         stats_window_size: int = 100,
-        tensorboard_log: Optional[str] = None,
-        policy_kwargs: Optional[dict[str, Any]] = None,
+        tensorboard_log: str | None = None,
+        policy_kwargs: dict[str, Any] | None = None,
         verbose: int = 0,
-        seed: Optional[int] = None,
-        device: Union[th.device, str] = "auto",
+        seed: int | None = None,
+        device: th.device | str = "auto",
         _init_setup_model: bool = True,
     ):
         super().__init__(