Introduction

Agent Reinforcement Trainer (ART) by OpenPipe is an open-source reinforcement learning (RL) library for training multi-step agents powered by large language models (LLMs). ART integrates RL training loops using the GRPO algorithm, enabling agents to learn from real interactions and improve performance in real-world tasks. Its unique client-server architecture offloads training complexity, allowing flexible deployment on local GPUs or ephemeral cloud backends with SkyPilot.

Key Features

• GRPO RL Algorithm: Specialized reinforcement learning for multi-turn LLM agents.
• Flexible Backend: Run training on local GPUs or use cloud resources with SkyPilot.
• Wide Model Support: Works with vLLM/HuggingFace transformers, including Qwen2.5, Qwen3, Llama.
• OpenAI-Compatible Client: Easy integration with your applications.
• Experiment Tracking: Integrates with Weights & Biases for detailed monitoring and model versioning.
• Example Notebooks: Walk-throughs for tasks like Tic Tac Toe, Codenames, and more.
• Permissive License: Apache-2.0 grants freedom for commercial use.

Installation Guide

ART is simple to install in a Python environment. Use pip to get started:

pip install openpipe-art

For SkyPilot backend support on cloud GPUs:

pip install openpipe-art[skypilot]

Sync dependencies for advanced features:

uv sync

After setup, ART is ready for RL training tasks and integration into your codebase.

How to Use

ART splits training between your code (client) and an external backend server (local or cloud). The workflow:

• The client gathers agent trajectory data during task runs.
• Assigns rewards to trajectories to signal learning quality.
• Sends batches for backend training using GRPO.
• The backend updates model weights via LoRA adapters and serves improved models.

This loop continues until your agent meets performance benchmarks.

Code Examples

Get started with this minimal example:

import art
Initialize trainable model
model = art.TrainableModel(name="agent-001", project="demo-art", base_model="Qwen/Qwen2.5-3B")

Setup backend (local GPU)
backend = art.LocalBackend()

Register model
await model.register(backend)

Train with rollouts
await model.train(rollouts)

For SkyPilot cloud backend:

from art.skypilot import SkyPilotBackend backend = await SkyPilotBackend.initialize_cluster(cluster_name="art-demo", gpu="A10") await model.register(backend) 

Contribution Guide

Contributions are welcome! The workflow:

• Fork ART repository.
• Make a new feature or fix branch.
• Sync dependencies with uv sync.
• Test and format code (./scripts/run_checks.sh).
• Submit a pull request—include a clear changelog entry.

Remote SSH via VSCode is supported for development on cluster GPUs.

Community & Support

• GitHub Issues Page for reporting bugs and asking questions.
• Track experiments and updates on Weights & Biases.
• Read official docs and browse example notebooks at ART Documentation.

OpenPipe actively maintains ART and welcomes new contributors.

Conclusion

ART delivers a scalable, flexible RL framework for agentic language models, supporting both local and cloud workflows. Its integration options, tracked experiments, and vibrant community ensure that developers can build cutting-edge LLM agents quickly and confidently.

Resources

• ART GitHub Repository
• Documentation
• Contribution Guidelines