Claude Skill

alterlab-stable-baselines3

Trains single-agent reinforcement learning agents with Stable-Baselines3 — PPO, SAC, DQN, TD3, DDPG, and A2C behind a scikit-learn-like API. Use for standard single-agent RL experiments, quick prototyping, well-documented algorithm implementations on Gymnasium environments, or ad

LLM Mart · 0 points · 1 views 0 listing impressions 0 install-command copies
Download AlterLab-IEU-AlterLab-Academic-Skills-skills_data-science_alterlab-stable-baselines3-3ab6d03.zip · 32 KB

Install

skills CLI npx skills add https://github.com/AlterLab-IEU/AlterLab-Academic-Skills/tree/main/skills/data-science/alterlab-stable-baselines3
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install alterlab-ieu-alterlab-academic-skills@llmmart
Git git clone https://github.com/AlterLab-IEU/AlterLab-Academic-Skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole alterlab-ieu/alterlab-academic-skills collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Stable Baselines3

Overview

Stable Baselines3 (SB3) is a PyTorch-based library providing reliable implementations of reinforcement learning algorithms. This skill provides comprehensive guidance for training RL agents, creating custom environments, implementing callbacks, and optimizing training workflows using SB3's unified API.

When to Use This Skill

Use this skill when the user wants to:

  • Train and evaluate a single-agent RL policy (PPO, A2C, SAC, TD3, DDPG, DQN, HER) on a Gymnasium environment.
  • Wrap a research simulator as a custom gymnasium.Env and validate it with check_env.
  • Add evaluation, checkpointing, early stopping, or custom logging through callbacks.
  • Speed up training with vectorized environments, or save, load, and report trained agents reproducibly.

Does NOT Trigger

Scenario Use Instead
Millions of steps per second, native multi-agent environments, or PufferEnv/Atari-scale throughput alterlab-pufferlib
Supervised deep-learning training loops (LightningModule, multi-GPU strategies) alterlab-pytorch-lightning
Black-box or multi-objective optimization of parameters without a sequential decision process alterlab-pymoo
Simulating queues or resource contention without learning a policy alterlab-simpy

Hand-off rule. When the request matches a row above, open your answer by naming that skill by its exact id (for example alterlab-pufferlib) and saying in one line why it fits better, then stop applying this skill's procedure. Name it even when you also need to ask for missing input. If only part of the request is out of scope, name the skill for that part and do only the in-scope part. A brief general pointer is fine; the named skill does the work.

Core Capabilities

1. Training RL Agents

Basic Training Pattern:

import gymnasium as gym
from stable_baselines3 import PPO

# Create environment
env = gym.make("CartPole-v1")

# Initialize agent
model = PPO("MlpPolicy", env, verbose=1)

# Train the agent
model.learn(total_timesteps=10000)

# Save the model
model.save("ppo_cartpole")

# Load the model (without prior instantiation)
model = PPO.load("ppo_cartpole", env=env)

Important Notes:

  • total_timesteps is a lower bound; actual training may exceed this due to batch collection
  • Use model.load() as a static method, not on an existing instance
  • The replay buffer is NOT saved with the model to save space

Algorithm Selection: Use references/algorithms.md for detailed algorithm characteristics and selection guidance. Quick reference:

  • PPO/A2C: General-purpose, supports all action space types, good for multiprocessing
  • SAC/TD3: Continuous control, off-policy, sample-efficient
  • DQN: Discrete actions, off-policy
  • HER: Goal-conditioned tasks

See scripts/train_rl_agent.py for a complete training template with best practices.

2. Custom Environments

Requirements: Custom environments must inherit from gymnasium.Env and implement:

  • __init__(): Define action_space and observation_space
  • reset(seed, options): Return initial observation and info dict
  • step(action): Return observation, reward, terminated, truncated, info
  • render(): Visualization (optional)
  • close(): Cleanup resources

Key Constraints:

  • Image observations must be np.uint8 in range [0, 255]
  • Use channel-first format when possible (channels, height, width)
  • SB3 normalizes images automatically by dividing by 255
  • Set normalize_images=False in policy_kwargs if pre-normalized
  • SB3 does NOT support Discrete or MultiDiscrete spaces with start!=0

Validation:

from stable_baselines3.common.env_checker import check_env

check_env(env, warn=True)

See scripts/custom_env_template.py for a complete custom environment template and references/custom_environments.md for comprehensive guidance.

3. Vectorized Environments

Purpose: Vectorized environments run multiple environment instances in parallel, accelerating training and enabling certain wrappers (frame-stacking, normalization).

Types:

  • DummyVecEnv: Sequential execution on current process (for lightweight environments)
  • SubprocVecEnv: Parallel execution across processes (for compute-heavy environments)

Quick Setup:

from stable_baselines3.common.env_util import make_vec_env
from stable_baselines3.common.vec_env import SubprocVecEnv

# Create 4 parallel environments
env = make_vec_env("CartPole-v1", n_envs=4, vec_env_cls=SubprocVecEnv)

model = PPO("MlpPolicy", env, verbose=1)
model.learn(total_timesteps=25000)

With SubprocVecEnv, wrap the call site in if __name__ == "__main__": (required on macOS/Windows spawn).

Off-Policy Optimization: When using multiple environments with off-policy algorithms (SAC, TD3, DQN), set gradient_steps=-1 to perform one gradient update per environment step, balancing wall-clock time and sample efficiency.

API Differences:

  • reset() returns only observations (info available in vec_env.reset_infos)
  • step() returns 4-tuple: (obs, rewards, dones, infos) not 5-tuple
  • Environments auto-reset after episodes
  • Terminal observations available via infos[env_idx]["terminal_observation"]

See references/vectorized_envs.md for detailed information on wrappers and advanced usage.

4. Callbacks for Monitoring and Control

Purpose: Callbacks enable monitoring metrics, saving checkpoints, implementing early stopping, and custom training logic without modifying core algorithms.

Common Callbacks:

  • EvalCallback: Evaluate periodically and save best model
  • CheckpointCallback: Save model checkpoints at intervals
  • StopTrainingOnRewardThreshold: Stop when target reward reached
  • ProgressBarCallback: Display training progress with timing

Custom Callback Structure:

from stable_baselines3.common.callbacks import BaseCallback

class CustomCallback(BaseCallback):
    def _on_training_start(self):
        # Called before first rollout
        pass

    def _on_step(self):
        # Called after each environment step
        # Return False to stop training
        return True

    def _on_rollout_end(self):
        # Called at end of rollout
        pass

Available Attributes:

  • self.model: The RL algorithm instance
  • self.num_timesteps: Total environment steps
  • self.training_env: The training environment

Chaining Callbacks:

from stable_baselines3.common.callbacks import CallbackList

callback = CallbackList([eval_callback, checkpoint_callback, custom_callback])
model.learn(total_timesteps=10000, callback=callback)

See references/callbacks.md for comprehensive callback documentation.

5. Model Persistence and Inspection

Saving and Loading:

# Save model
model.save("model_name")

# Save normalization statistics (if using VecNormalize)
vec_env.save("vec_normalize.pkl")

# Load model
model = PPO.load("model_name", env=env)

# Load normalization statistics
vec_env = VecNormalize.load("vec_normalize.pkl", vec_env)

Parameter Access:

# Get parameters
params = model.get_parameters()

# Set parameters
model.set_parameters(params)

# Access PyTorch state dict
state_dict = model.policy.state_dict()

6. Evaluation and Recording

Evaluation:

from stable_baselines3.common.evaluation import evaluate_policy

mean_reward, std_reward = evaluate_policy(
    model,
    env,
    n_eval_episodes=10,
    deterministic=True
)

Video Recording (the wrapped env must be created with render_mode="rgb_array", and moviepy must be importable, e.g. add --with "gymnasium[other]" to the uv run command):

from stable_baselines3.common.vec_env import VecVideoRecorder

# Wrap environment with video recorder
env = VecVideoRecorder(
    env,
    "videos/",
    record_video_trigger=lambda x: x % 2000 == 0,
    video_length=200
)

See scripts/evaluate_agent.py for a complete evaluation and recording template.

7. Advanced Features

Learning Rate Schedules:

def linear_schedule(initial_value):
    def func(progress_remaining):
        # progress_remaining goes from 1 to 0
        return progress_remaining * initial_value
    return func

model = PPO("MlpPolicy", env, learning_rate=linear_schedule(0.001))

# Built-in equivalent (SB3 >= 2.7; replaces the deprecated get_linear_fn/constant_fn/get_schedule_fn)
from stable_baselines3.common.utils import LinearSchedule
model = PPO("MlpPolicy", env, learning_rate=LinearSchedule(1e-3, 0.0, 1.0))

N-step Returns (off-policy, SB3 >= 2.7): SAC, TD3, DDPG, and DQN accept n_steps= to bootstrap from n-step returns (backed by NStepReplayBuffer).

Multi-Input Policies (Dict Observations):

model = PPO("MultiInputPolicy", env, verbose=1)

Use when observations are dictionaries (e.g., combining images with sensor data).

Hindsight Experience Replay:

from stable_baselines3 import SAC, HerReplayBuffer

model = SAC(
    "MultiInputPolicy",
    env,
    replay_buffer_class=HerReplayBuffer,
    replay_buffer_kwargs=dict(
        n_sampled_goal=4,
        goal_selection_strategy="future",
    ),
)

TensorBoard Integration:

model = PPO("MlpPolicy", env, tensorboard_log="./tensorboard/")
model.learn(total_timesteps=10000)

Workflow Guidance

Starting a New RL Project:

  1. Define the problem: Identify observation space, action space, and reward structure
  2. Choose algorithm: Use references/algorithms.md for selection guidance
  3. Create/adapt environment: Use scripts/custom_env_template.py if needed
  4. Validate environment: Always run check_env() before training
  5. Set up training: Use scripts/train_rl_agent.py as starting template
  6. Add monitoring: Implement callbacks for evaluation and checkpointing
  7. Optimize performance: Consider vectorized environments for speed
  8. Evaluate and iterate: Use scripts/evaluate_agent.py for assessment

Common Issues:

  • Memory errors: Reduce buffer_size for off-policy algorithms or use fewer parallel environments
  • Slow training: Consider SubprocVecEnv for parallel environments
  • Unstable training: Try different algorithms, tune hyperparameters, or check reward scaling
  • Import errors: Run through uv instead of installing: uv run --python ">=3.12" --with "stable-baselines3[extra]>=2.0" python train.py

Resources

scripts/

  • train_rl_agent.py: Complete training script template with best practices
  • evaluate_agent.py: Agent evaluation and video recording template
  • custom_env_template.py: Custom Gym environment template

references/

  • algorithms.md: Detailed algorithm comparison and selection guide
  • custom_environments.md: Comprehensive custom environment creation guide
  • callbacks.md: Complete callback system reference
  • vectorized_envs.md: Vectorized environment usage and wrappers

Installation

Requires Python 3.10+ and PyTorch >= 2.8 (SB3 2.9 raised the floor from 2.3). The [extra] option pulls in TensorBoard, OpenCV, ale-py (Atari), pygame-ce, tqdm/rich (progress bar), and — since 2.9 — pandas and matplotlib, which are no longer core dependencies (needed for results_plotter/load_results). Atari IDs such as "PongNoFrameskip-v4" or "ALE/Pong-v5" are registered when ale_py is imported (import ale_py; gym.register_envs(ale_py)).

# No install step: uv resolves SB3 (and PyTorch) per call
uv run --python ">=3.12" --with "stable-baselines3>=2.0" python train.py

# With extra dependencies (Tensorboard, Atari, etc.)
uv run --python ">=3.12" --with "stable-baselines3[extra]>=2.0" python train.py

# SB3 already importable (`python -c "import stable_baselines3"`)? run with plain `python`.
# CPU-only machine: add --index https://download.pytorch.org/whl/cpu so uv fetches CPU torch, not multi-GB CUDA wheels

Part of the AlterLab Academic Skills suite.

Files (alterlab-academic-skills)
  • evals
    • fixtures
      • inventory_env.py 2.5 KB
        Loading…
    • evals.json 6.1 KB
      Loading…
  • references
    • algorithms.md 10.1 KB
      Loading…
    • callbacks.md 15.3 KB
      Loading…
    • custom_environments.md 13.4 KB
      Loading…
    • vectorized_envs.md 14.8 KB
      Loading…
  • scripts
    • custom_env_template.py 9.1 KB
      Loading…
    • evaluate_agent.py 7.7 KB
      Loading…
    • train_rl_agent.py 5 KB
      Loading…
  • SKILL.md 12.8 KB
    ---
    name: alterlab-stable-baselines3
    description: Trains single-agent reinforcement learning agents with Stable-Baselines3 — PPO, SAC, DQN, TD3, DDPG, and A2C behind a scikit-learn-like API. Use for standard single-agent RL experiments, quick prototyping, well-documented algorithm implementations on Gymnasium environments, or adding callbacks and evaluation. For high-throughput parallel training, multi-agent systems, or custom vectorized environments prefer alterlab-pufferlib. Part of the AlterLab Academic Skills suite.
    license: MIT
    allowed-tools: Read Write Edit Bash(python:*) Bash(uv:*)
    compatibility: No API key required. Runs locally via `uv run --python ">=3.12" --with "stable-baselines3[extra]>=2.0" python script.py` (no install step; the interpreter pin matters because an older default Python resolves older releases); requires stable-baselines3 >= 2.0 (current 2.9.0 as of 2026-09, which needs Python >= 3.10, PyTorch >= 2.8, and gymnasium >= 0.29.1,<2.0; current gymnasium 1.3).
    metadata:
        skill-author: AlterLab
        version: "1.0.1"
        last_updated: "2026-09-23"
    ---
    
    # Stable Baselines3
    
    ## Overview
    
    Stable Baselines3 (SB3) is a PyTorch-based library providing reliable implementations of reinforcement learning algorithms. This skill provides comprehensive guidance for training RL agents, creating custom environments, implementing callbacks, and optimizing training workflows using SB3's unified API.
    
    ## When to Use This Skill
    
    Use this skill when the user wants to:
    - Train and evaluate a single-agent RL policy (PPO, A2C, SAC, TD3, DDPG, DQN, HER) on a Gymnasium environment.
    - Wrap a research simulator as a custom `gymnasium.Env` and validate it with `check_env`.
    - Add evaluation, checkpointing, early stopping, or custom logging through callbacks.
    - Speed up training with vectorized environments, or save, load, and report trained agents reproducibly.
    
    ### Does NOT Trigger
    
    | Scenario | Use Instead |
    |----------|-------------|
    | Millions of steps per second, native multi-agent environments, or PufferEnv/Atari-scale throughput | `alterlab-pufferlib` |
    | Supervised deep-learning training loops (LightningModule, multi-GPU strategies) | `alterlab-pytorch-lightning` |
    | Black-box or multi-objective optimization of parameters without a sequential decision process | `alterlab-pymoo` |
    | Simulating queues or resource contention without learning a policy | `alterlab-simpy` |
    
    **Hand-off rule.** When the request matches a row above, open your answer by naming that skill by its exact id (for example `alterlab-pufferlib`) and saying in one line why it fits better, then stop applying this skill's procedure. Name it even when you also need to ask for missing input. If only part of the request is out of scope, name the skill for that part and do only the in-scope part. A brief general pointer is fine; the named skill does the work.
    
    ## Core Capabilities
    
    ### 1. Training RL Agents
    
    **Basic Training Pattern:**
    
    ```python
    import gymnasium as gym
    from stable_baselines3 import PPO
    
    # Create environment
    env = gym.make("CartPole-v1")
    
    # Initialize agent
    model = PPO("MlpPolicy", env, verbose=1)
    
    # Train the agent
    model.learn(total_timesteps=10000)
    
    # Save the model
    model.save("ppo_cartpole")
    
    # Load the model (without prior instantiation)
    model = PPO.load("ppo_cartpole", env=env)
    ```
    
    **Important Notes:**
    - `total_timesteps` is a lower bound; actual training may exceed this due to batch collection
    - Use `model.load()` as a static method, not on an existing instance
    - The replay buffer is NOT saved with the model to save space
    
    **Algorithm Selection:**
    Use `references/algorithms.md` for detailed algorithm characteristics and selection guidance. Quick reference:
    - **PPO/A2C**: General-purpose, supports all action space types, good for multiprocessing
    - **SAC/TD3**: Continuous control, off-policy, sample-efficient
    - **DQN**: Discrete actions, off-policy
    - **HER**: Goal-conditioned tasks
    
    See `scripts/train_rl_agent.py` for a complete training template with best practices.
    
    ### 2. Custom Environments
    
    **Requirements:**
    Custom environments must inherit from `gymnasium.Env` and implement:
    - `__init__()`: Define action_space and observation_space
    - `reset(seed, options)`: Return initial observation and info dict
    - `step(action)`: Return observation, reward, terminated, truncated, info
    - `render()`: Visualization (optional)
    - `close()`: Cleanup resources
    
    **Key Constraints:**
    - Image observations must be `np.uint8` in range [0, 255]
    - Use channel-first format when possible (channels, height, width)
    - SB3 normalizes images automatically by dividing by 255
    - Set `normalize_images=False` in policy_kwargs if pre-normalized
    - SB3 does NOT support `Discrete` or `MultiDiscrete` spaces with `start!=0`
    
    **Validation:**
    ```python
    from stable_baselines3.common.env_checker import check_env
    
    check_env(env, warn=True)
    ```
    
    See `scripts/custom_env_template.py` for a complete custom environment template and `references/custom_environments.md` for comprehensive guidance.
    
    ### 3. Vectorized Environments
    
    **Purpose:**
    Vectorized environments run multiple environment instances in parallel, accelerating training and enabling certain wrappers (frame-stacking, normalization).
    
    **Types:**
    - **DummyVecEnv**: Sequential execution on current process (for lightweight environments)
    - **SubprocVecEnv**: Parallel execution across processes (for compute-heavy environments)
    
    **Quick Setup:**
    ```python
    from stable_baselines3.common.env_util import make_vec_env
    from stable_baselines3.common.vec_env import SubprocVecEnv
    
    # Create 4 parallel environments
    env = make_vec_env("CartPole-v1", n_envs=4, vec_env_cls=SubprocVecEnv)
    
    model = PPO("MlpPolicy", env, verbose=1)
    model.learn(total_timesteps=25000)
    ```
    With `SubprocVecEnv`, wrap the call site in `if __name__ == "__main__":` (required on macOS/Windows spawn).
    
    **Off-Policy Optimization:**
    When using multiple environments with off-policy algorithms (SAC, TD3, DQN), set `gradient_steps=-1` to perform one gradient update per environment step, balancing wall-clock time and sample efficiency.
    
    **API Differences:**
    - `reset()` returns only observations (info available in `vec_env.reset_infos`)
    - `step()` returns 4-tuple: `(obs, rewards, dones, infos)` not 5-tuple
    - Environments auto-reset after episodes
    - Terminal observations available via `infos[env_idx]["terminal_observation"]`
    
    See `references/vectorized_envs.md` for detailed information on wrappers and advanced usage.
    
    ### 4. Callbacks for Monitoring and Control
    
    **Purpose:**
    Callbacks enable monitoring metrics, saving checkpoints, implementing early stopping, and custom training logic without modifying core algorithms.
    
    **Common Callbacks:**
    - **EvalCallback**: Evaluate periodically and save best model
    - **CheckpointCallback**: Save model checkpoints at intervals
    - **StopTrainingOnRewardThreshold**: Stop when target reward reached
    - **ProgressBarCallback**: Display training progress with timing
    
    **Custom Callback Structure:**
    ```python
    from stable_baselines3.common.callbacks import BaseCallback
    
    class CustomCallback(BaseCallback):
        def _on_training_start(self):
            # Called before first rollout
            pass
    
        def _on_step(self):
            # Called after each environment step
            # Return False to stop training
            return True
    
        def _on_rollout_end(self):
            # Called at end of rollout
            pass
    ```
    
    **Available Attributes:**
    - `self.model`: The RL algorithm instance
    - `self.num_timesteps`: Total environment steps
    - `self.training_env`: The training environment
    
    **Chaining Callbacks:**
    ```python
    from stable_baselines3.common.callbacks import CallbackList
    
    callback = CallbackList([eval_callback, checkpoint_callback, custom_callback])
    model.learn(total_timesteps=10000, callback=callback)
    ```
    
    See `references/callbacks.md` for comprehensive callback documentation.
    
    ### 5. Model Persistence and Inspection
    
    **Saving and Loading:**
    ```python
    # Save model
    model.save("model_name")
    
    # Save normalization statistics (if using VecNormalize)
    vec_env.save("vec_normalize.pkl")
    
    # Load model
    model = PPO.load("model_name", env=env)
    
    # Load normalization statistics
    vec_env = VecNormalize.load("vec_normalize.pkl", vec_env)
    ```
    
    **Parameter Access:**
    ```python
    # Get parameters
    params = model.get_parameters()
    
    # Set parameters
    model.set_parameters(params)
    
    # Access PyTorch state dict
    state_dict = model.policy.state_dict()
    ```
    
    ### 6. Evaluation and Recording
    
    **Evaluation:**
    ```python
    from stable_baselines3.common.evaluation import evaluate_policy
    
    mean_reward, std_reward = evaluate_policy(
        model,
        env,
        n_eval_episodes=10,
        deterministic=True
    )
    ```
    
    **Video Recording** (the wrapped env must be created with `render_mode="rgb_array"`, and `moviepy` must be importable, e.g. add `--with "gymnasium[other]"` to the uv run command):
    ```python
    from stable_baselines3.common.vec_env import VecVideoRecorder
    
    # Wrap environment with video recorder
    env = VecVideoRecorder(
        env,
        "videos/",
        record_video_trigger=lambda x: x % 2000 == 0,
        video_length=200
    )
    ```
    
    See `scripts/evaluate_agent.py` for a complete evaluation and recording template.
    
    ### 7. Advanced Features
    
    **Learning Rate Schedules:**
    ```python
    def linear_schedule(initial_value):
        def func(progress_remaining):
            # progress_remaining goes from 1 to 0
            return progress_remaining * initial_value
        return func
    
    model = PPO("MlpPolicy", env, learning_rate=linear_schedule(0.001))
    
    # Built-in equivalent (SB3 >= 2.7; replaces the deprecated get_linear_fn/constant_fn/get_schedule_fn)
    from stable_baselines3.common.utils import LinearSchedule
    model = PPO("MlpPolicy", env, learning_rate=LinearSchedule(1e-3, 0.0, 1.0))
    ```
    
    **N-step Returns (off-policy, SB3 >= 2.7):** SAC, TD3, DDPG, and DQN accept `n_steps=` to bootstrap from n-step returns (backed by `NStepReplayBuffer`).
    
    **Multi-Input Policies (Dict Observations):**
    ```python
    model = PPO("MultiInputPolicy", env, verbose=1)
    ```
    Use when observations are dictionaries (e.g., combining images with sensor data).
    
    **Hindsight Experience Replay:**
    ```python
    from stable_baselines3 import SAC, HerReplayBuffer
    
    model = SAC(
        "MultiInputPolicy",
        env,
        replay_buffer_class=HerReplayBuffer,
        replay_buffer_kwargs=dict(
            n_sampled_goal=4,
            goal_selection_strategy="future",
        ),
    )
    ```
    
    **TensorBoard Integration:**
    ```python
    model = PPO("MlpPolicy", env, tensorboard_log="./tensorboard/")
    model.learn(total_timesteps=10000)
    ```
    
    ## Workflow Guidance
    
    **Starting a New RL Project:**
    
    1. **Define the problem**: Identify observation space, action space, and reward structure
    2. **Choose algorithm**: Use `references/algorithms.md` for selection guidance
    3. **Create/adapt environment**: Use `scripts/custom_env_template.py` if needed
    4. **Validate environment**: Always run `check_env()` before training
    5. **Set up training**: Use `scripts/train_rl_agent.py` as starting template
    6. **Add monitoring**: Implement callbacks for evaluation and checkpointing
    7. **Optimize performance**: Consider vectorized environments for speed
    8. **Evaluate and iterate**: Use `scripts/evaluate_agent.py` for assessment
    
    **Common Issues:**
    
    - **Memory errors**: Reduce `buffer_size` for off-policy algorithms or use fewer parallel environments
    - **Slow training**: Consider SubprocVecEnv for parallel environments
    - **Unstable training**: Try different algorithms, tune hyperparameters, or check reward scaling
    - **Import errors**: Run through uv instead of installing: `uv run --python ">=3.12" --with "stable-baselines3[extra]>=2.0" python train.py`
    
    ## Resources
    
    ### scripts/
    - `train_rl_agent.py`: Complete training script template with best practices
    - `evaluate_agent.py`: Agent evaluation and video recording template
    - `custom_env_template.py`: Custom Gym environment template
    
    ### references/
    - `algorithms.md`: Detailed algorithm comparison and selection guide
    - `custom_environments.md`: Comprehensive custom environment creation guide
    - `callbacks.md`: Complete callback system reference
    - `vectorized_envs.md`: Vectorized environment usage and wrappers
    
    ## Installation
    
    Requires Python 3.10+ and PyTorch >= 2.8 (SB3 2.9 raised the floor from 2.3). The `[extra]` option pulls in TensorBoard, OpenCV, `ale-py` (Atari), `pygame-ce`, tqdm/rich (progress bar), and — since 2.9 — pandas and matplotlib, which are no longer core dependencies (needed for `results_plotter`/`load_results`). Atari IDs such as `"PongNoFrameskip-v4"` or `"ALE/Pong-v5"` are registered when `ale_py` is imported (`import ale_py; gym.register_envs(ale_py)`).
    
    ```bash
    # No install step: uv resolves SB3 (and PyTorch) per call
    uv run --python ">=3.12" --with "stable-baselines3>=2.0" python train.py
    
    # With extra dependencies (Tensorboard, Atari, etc.)
    uv run --python ">=3.12" --with "stable-baselines3[extra]>=2.0" python train.py
    
    # SB3 already importable (`python -c "import stable_baselines3"`)? run with plain `python`.
    # CPU-only machine: add --index https://download.pytorch.org/whl/cpu so uv fetches CPU torch, not multi-GB CUDA wheels
    ```
    
    Part of the AlterLab Academic Skills suite.
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.