Agents That Improve Through Use: AReaL 2.0 Open-Sources a 'Growth System' for Intelligent Agents

On July 2, AReaL—the open-source reinforcement learning infrastructure project—officially released version 2.0. AReaL is designed to connect foundation model training with modern agent applications, delivering efficient reinforcement learning support for Agent-driven scenarios.

AReaL 2.0 is engineered for Agents already operating in live business environments, providing infrastructure for continuous in-use learning. Through AReaL 2.0, interaction traces generated during real task execution are recorded, structured, and routed into subsequent training workflows, enabling sustained model refinement. The result: Agents that grow stronger through use, within a controlled safety boundary.

Agents are now operating in live production environments—writing code, retrieving data, invoking tools, and executing increasingly sophisticated tasks within enterprise systems. Yet a fundamental problem persists: Agents work daily, but derive little growth from their labor.

In production, Agents accumulate substantial experiential data: which tasks were executed successfully, where tool invocations failed, why user satisfaction was compromised, and whether a given decision veered off course. Yet the bulk of this information is archived as logs, rarely transformed reliably or securely into measurable capability gains.

AReaL 2.0 addresses precisely this question: how deployed Agents continue to improve. Developers need not rebuild their Agents; by routing Agent-to-LLM requests through AReaL 2.0's unified inference gateway, the online reinforcement learning pipeline is engaged.

Caption: AReaL 2.0 Online Reinforcement Learning (Online RL) architecture schematic

Consider Hermes Agent: Hermes continues to receive tasks, plan steps, and invoke models as normal, while AReaL 2.0 silently records critical interactions during task execution. Post-task feedback and reward signals are integrated, and the resulting real-world trajectories are fed into downstream training. Developers may substitute Hermes with their own Agent and task environment, constructing an identical online RL pipeline.

Agent capability enhancement is thus no longer tethered to manually curated data, offline training, and redeployment cycles. Multi-turn dialogues, tool invocations, execution outcomes, and feedback signals from live tasks all become viable feedstock for continued model learning.

This is particularly critical in enterprise settings. Agents embedded in corporate workflows contend with real, complex, and ever-evolving tasks: codebases are updated, processes are reconfigured, user requirements shift, and tools and systems undergo change. An Agent whose capabilities are frozen upon deployment cannot adapt sustainably. AReaL 2.0 targets precisely this gap—the missing link between tool proficiency and learning from tool usage.

Continuous learning in production environments cannot be reduced to "collect and retrain." Agents may handle code, customer data, enterprise knowledge bases, and internal systems; the training pipeline must thus accommodate access control, data sanitization, isolation, and audit requirements. AReaL 2.0 introduces a trajectory data proxy mechanism, designed to ensure that real-world task data is governed and utilized under enhanced security and control when it enters the training pipeline.

The AReaL team's technical report identifies the critical bottleneck for self-evolving Agents: neither model capability alone nor reinforcement learning sophistication is determinative. The missing piece is an online RL infrastructure purpose-built for production Agents. AReaL 2.0 represents an architectural upgrade targeting next-generation agent applications—bridging Agent services, real task trajectories, data governance, and online RL training into a deployable engineering foundation for post-deployment learning.

Viewed from a longer horizon, AReaL 2.0 signals a paradigm shift in next-generation agent applications. No longer are Agents single-shot tools—trained, deployed, and frozen. They become entities that harvest feedback continuously in live environments, transmute success and failure alike into experience, and advance their capabilities within a defined safety perimeter.

AReaL was founded in 2024 by teams from Ant Group, Tsinghua University, and the Hong Kong University of Science and Technology. In May 2026, AReaL graduated from Ant Group's InclusionAI incubation as an independent open-source community and was admitted to the PyTorch Foundation Ecosystem, deepening its integration into the mainstream reinforcement learning infrastructure landscape.

As the community charts its independent course, AReaL continues to attract participation and support from industry and open-source partners, including Huawei Cloud and MindLab, among others. Looking ahead, AReaL will iterate on online RL, automated evaluation, and multimodal agent training, collaborating with the community to advance the self-evolving agent ecosystem.

The AReaL 2.0 technical report and codebase have been released as open source.

GitHub repository: https://github.com/areal-project/AReaL

Technical report: https://arxiv.org/abs/2607.01120