Introduction

Developing multi-agent systems often requires a standardized way to simulate interactions between multiple AI agents, but most reinforcement learning tools are built for single-agent scenarios. PettingZoo, a Python library with over 3.5k GitHub stars, fills this gap by providing a unified API for multi-agent reinforcement learning (MARL) environments. It effectively serves as the multi-agent equivalent of Gymnasium (formerly OpenAI Gym), allowing researchers and developers to benchmark and train agents in complex, interactive simulations.

What Is PettingZoo?

PettingZoo is a Python library that provides a standardized API for multi-agent reinforcement learning environments for researchers and developers. Maintained by the Farama Foundation, it offers a wide variety of reference environments and utilities to create custom MARL tasks. Licensed under the MIT and Apache 2.0 licenses, it is designed to be highly scalable and user-friendly, mimicking the simplicity of Gymnasium but adapted for the complexities of multi-agent interactions.

The library is built around two primary interaction models: the Agent Environment Cycle (AEC) for sequential, turn-based interactions and the Parallel API for simultaneous actions. This dual-API approach allows PettingZoo to represent a broad spectrum of games and simulations, from classic board games to highly coordinated graphical simulations.

Why PettingZoo Matters

Before PettingZoo, the MARL community lacked a consistent interface. Researchers often had to write custom wrappers for every new environment, leading to fragmented results and difficulty in reproducing experiments. This fragmentation made it nearly impossible to swap agents or algorithms across different tasks without significant code rewrites.

PettingZoo solves this by introducing a universal API. By standardizing how observations, actions, and rewards are handled across different environments, it enables the MARL community to create interchangeable agents. This acceleration of research is akin to what OpenAI Gym did for single-agent RL, providing a common language for benchmarking and algorithm development.

With growing interest in multi-agent coordination, competitive AI, and swarm intelligence, PettingZoo has become a critical piece of infrastructure for anyone moving beyond single-agent AI. Its integration with popular libraries like Ray RLlib, Tianshou, and CleanRL makes it the industry standard for MARL environment management.

Key Features

• Agent Environment Cycle (AEC) API: A novel interaction model that supports sequential turn-based environments, ensuring that agents act in a clean, predictable cycle.
• Parallel API: A high-performance interface for environments where agents act simultaneously, making it ideal for real-time simulations and vectorized training.
• Diverse Environment Families: Includes a vast array of tasks, such as Atari (multiplayer Atari 2600 games), Butterfly (highly coordinated cooperative graphics games), Classic (board and card games), and MPE (Multi-Agent Particle Environments).
• Gymnasium Compatibility: Built to be a drop-in multi-agent extension of the Gymnasium API, ensuring a low learning curve for those already familiar with single-agent RL.
• Strict Versioning: Implements a strict versioning system (e.g., _v0, _v1) for all environments to ensure reproducibility in research papers and benchmarks.
• Custom Environment Tools: Provides comprehensive utilities and tutorials for developers to build their own compliant MARL environments from scratch.
• Extensive Integration: Seamlessly works with leading MARL frameworks such as Ray RLlib, AgileRL, and Sample Factory.
• Cross-Platform Support: Officially supports Linux and macOS, with community-driven support for Windows via WSL.

How PettingZoo Compares

PettingZoo is the clear choice for researchers who need a Gymnasium-like experience but for multiple agents. While OpenAI Gym/Gymnasium is the gold standard for single-agent RL, it cannot natively handle the complexities of agent death, agent creation, or turn-based cycles without cumbersome wrappers. OpenSpiel is powerful for game theory research, but its API is significantly more complex and often less intuitive for those coming from a deep reinforcement learning background.

The primary differentiator for PettingZoo is its AEC (Agent Environment Cycle) model. This model prevents common bugs associated with simultaneous action assumptions in turn-based games, providing a mathematically sound foundation for MARL research. For those requiring high-throughput training, the Parallel API provides the necessary performance without sacrificing the standardized interface.

Getting Started: Installation

PettingZoo is available via pip and supports Python 3.9 through 3.14 on Linux and macOS. Windows users are encouraged to use the Windows Subsystem for Linux (WSL).

Base Installation

To install the core library without additional environment dependencies:

pip install pettingzoo

Environment-Specific Installation

Because different environment families have different system requirements, you can install dependencies for specific families to keep your environment lean:

pip install 'pettingzoo[atari]'

Full Installation

To install PettingZoo along with all available environment dependencies:

pip install 'pettingzoo[all]'

Prerequisites: Some Linux distributions may require manual installation of system-level dependencies such as cmake, swig, or zlib1g-dev (e.g., sudo apt install cmake swig zlib1g-dev).

How to Use PettingZoo

Interacting with a PettingZoo environment follows a workflow similar to Gymnasium. You initialize the environment, reset it, and then enter a loop where agents take actions based on their observations.

For turn-based environments using the AEC API, the environment itself manages the order of agents. You use agent_iter() to loop through agents as the environment determines who is allowed to act.

In Parallel environments, all agents are processed simultaneously. You provide a dictionary of actions (one for each agent) and receive a dictionary of observations and rewards in return. This is the preferred method for most deep RL training pipelines.

Code Examples

Basic AEC Interaction Loop

This example demonstrates the standard turn-based interaction loop using the AEC API.

from pettingzoo.butterfly import knights_archers_zombies_v10

env = knights_archers_zombies_v10.env(render_mode="human")
env.reset(seed=42)

for agent in env.agent_iter():
    observation, reward, termination, truncation, info = env.last()
    
    if termination or truncation:
        action = None
    else:
        # Replace this with your policy
        action = env.action_space(agent).sample()
    
    env.step(action)

env.close()

Parallel API Usage

This example shows how to use the Parallel API for simultaneous agent actions, which is more common for training neural networks.

from pettingzoo.butterfly import pistonball_v6

env = pistonball_v6.parallel_env(render_mode="human")
env.reset(seed=42)

# Example loop for 2 steps
for _ in range(2):
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}
    observations, rewards, terminations, truncations, infos = env.step(actions)
    
    # Process observations and rewards for all agents
    for agent, obs in observations.items():
        print(f"Agent {agent} observation: {obs}")

env.close()

Real-World Use Cases

PettingZoo is widely used in academic research and industrial AI development to simulate complex multi-agent dynamics.

• Cooperative Robotics: Researchers use the Butterfly environment family to train agents that must coordinate their movements to achieve a common goal, such as moving an object together.
• Competitive Game AI: Developers use the Classic environment family (e.g., Tic-Tac-Toe, Connect Four) to benchmark new MARL algorithms against self-play opponents.
• Swarm Intelligence: The MPE (Multi-Agent Particle Environments) family is used to simulate swarm behavior, such as agents coordinating to surround a target or spread out across a landmark map.
• Financial Modeling: By creating custom environments, developers can simulate market participants as agents to study the emergent properties of algorithmic trading and market stability.

Contributing to PettingZoo

PettingZoo is an open-source project maintained by the Farama Foundation. Contributions are highly encouraged to expand the library of reference environments and improve the core API.

To contribute, start by forking the repository and creating a feature branch. The project follows standard GitHub flow: submit a pull request for review. If you are looking for a good first issue, check the GitHub Issues tab for labels like “good first issue” or “bug”.

The project maintains a strict code style guide (using Black) and requires all contributors to follow the MIT and Apache 2.0 licenses. Ensure your changes are documented in the documentation site at pettingzoo.farama.org.

Community and Support

PettingZoo has a vibrant community of MARL researchers and developers. The primary channel for coordination and support is the official Discord server, which is used for both development work and user troubleshooting.

Official documentation is hosted at pettingzoo.farama.org. For more formal technical discussions and bug reports, the GitHub repository is the central hub for issues and pull requests.

Conclusion

PettingZoo is the essential infrastructure for anyone serious about multi-agent reinforcement learning. By providing a standardized API and a diverse set of benchmark environments, it removes the friction of environment management and allows researchers to focus on algorithm development.

Whether you are building cooperative swarms, competitive game AI, or complex social simulations, PettingZoo provides the tools and reproducibility that academic and industrial research requires. It is the right choice when you need a standardized, Gymnasium-compatible interface for multiple agents.

Star the repo, try the quickstart, and join the community on Discord to start build the next generation of multi-agent AI.

Resources

• PettingZoo GitHub Repository
• Official PettingZoo Documentation
• PettingZoo Discord Server