Note

Ray 2.10.0 introduces the alpha stage of RLlib’s “new API stack”. The Ray Team plans to transition algorithms, example scripts, and documentation to the new code base thereby incrementally replacing the “old API stack” (e.g., ModelV2, Policy, RolloutWorker) throughout the subsequent minor releases leading up to Ray 3.0.

Note, however, that so far only PPO (single- and multi-agent) and SAC (single-agent only) support the “new API stack” and continue to run by default with the old APIs. You can continue to use the existing custom (old stack) classes.

See here for more details on how to use the new API stack.

Ray RLlib API

Tip

We’d love to hear your feedback on using RLlib - sign up to our forum and start asking questions!

This section contains an overview of RLlib’s package- and API reference. If you think there is anything missing, please open an issue on Github.

• Algorithms
  • Algorithm Configuration API
  • Building Custom Algorithm Classes
  • Algorithm API
• Environments
  • Environment API Reference
• Policy API
  • Building Custom Policy Classes
  • Base Policy classes
  • Making models
  • Inference
  • Computing, processing, and applying gradients
  • Updating the Policy’s model
  • Loss, Logging, optimizers, and trajectory processing
  • Saving and restoring
  • Connectors
  • Recurrent Policies
  • Miscellaneous
• Model APIs
  • Base Model classes
  • Feed Forward methods
  • Recurrent Models API
  • Acessing variables
  • Customization
• Catalog API
  • Basic usage
  • Advanced usage
• RLModule API
  • RL Module specifications and configurations
  • RL Module API
  • Multi Agent RL Module API
• LearnerGroup API
  • Configuring a LearnerGroup and Learner Workers
  • Constructing a LearnerGroup
• Learner API
  • Constructing a Learner
  • Performing Updates
  • Computing Losses
  • Configuring Optimizers
  • Gradient Computation
  • Saving, Loading, Checkpointing, and Restoring States
  • Adding and Removing Modules
• Sampling the Environment or offline data
  • RolloutWorker API
  • EnvRunner API
  • EnvRunnerGroup API
  • Sampler API
  • Policy Map API
  • Sample batch API
  • MultiAgent batch API
• Replay Buffer API
  • Replay Buffer Base Classes
  • Public Methods
  • Multi Agent Buffers
  • Utility Methods
• RLlib Utilities
  • Exploration API
  • Scheduler API
  • Training Operations Utilities
  • Framework Utilities
• External Application API
  • PolicyClient
  • PolicyServerInput