v2.0.0

🚀 Key Highlights

Micro-service Architecture

Includes training, inference, agent, and weight update services.

Hermes Agent Example

End-to-end agent built on the agent service and wired into the 2.0 training loop. Check out the Hermes RL training example for more details.

CLI Commands

A unified areal CLI to launch, manage, and operate each micro-service. See the per-service guides: training, inference, agent.

What's Changed

• perf(infra): pipeline controller initialization with background threads by @garrett4wade in #1294
• ci: add nightly workflow for long-running tests by @garrett4wade in #1312
• feat: support pp for Sglang by @TaoZex in #1162
• perf(archon): parallelize inference controller initialization by @garrett4wade in #1314
• fix(archon): improve HTTP client reliability and reduce logging verbosity by @garrett4wade in #1315
• refactor(experimental): remove redundant capacity grant from inference service by @garrett4wade in #1318
• refactor(experimental): add unified group semantics to inference service session lifecycle by @garrett4wade in #1321
• chore: migrate repo references from InclusionAI to areal-project by @garrett4wade in #1325
• fix(proxy): refuse default admin API key on non-loopback bind by @sebastiondev in #1323
• refactor: consolidate admin key validation into shared helper by @garrett4wade in #1328
• fix: Add error detection function and test for ZeroDivisionError and other errors alike by @chenzhiyi021 in #1332
• docs: Add OpenSSF Best Practices badge to README 📛 by @mingcheng in #1348
• gov: add new maintainer by @sitabulaixizawaluduo in #1349
• fix(utils): mask 2d sequence advantages by @haoyang9804 in #1346
• feat(awex): add colocated CUDA IPC weight transfer by @garrett4wade in #1310
• fix(infra): correct staleness capacity inflation after recovery by @daihaowz in #1345
• fix(test): add missing colocate field to wu_controller connect tests by @sitabulaixizawaluduo in #1357
• refactor(infra): backport rl infra cleanup by @sitabulaixizawaluduo in #1353
• feat(megatron): implement async_save with AsyncCallsQueue by @dingzhiqiang in #1356
• feat: controller v2 refactor by @sitabulaixizawaluduo in #1354
• fix(fsdp): maintain fp32 master weights for AdamW (#1292) by @guozhihao-224 in #1369
• chore: add 0516 community meeting materials and update agenda for the next biweekly sync by @sitabulaixizawaluduo in #1371
• docs: fix some typos by @jeis4wpi in #1352
• feat:enable v2 training pipeline with controller parity by @sitabulaixizawaluduo in #1363
• fix(trainer): skip controller-side CUDA sync in single-controller mode by @Adiactive in #1377
• fix[v2]: localize RTensor trajectories before reading on controller by @sitabulaixizawaluduo in #1387
• chore: migrate community governance files to external repository 📋 by @mingcheng in #1386
• feat(megatron): Qwen3.5 dense + MoE training/inference support via megatron-bridge by @Adiactive in #1384
• feat(distillation): add on-policy distillation using RolloutEngine by @zahrayousefijamarani in #1376
• docs(roadmap): add 2026 Q2 and H2 milestones by @sitabulaixizawaluduo in #1390
• Fix LoRA model training by @lifeiteng in #1385
• fix(v2/awex): unblock weight-update bring-up by @sitabulaixizawaluduo in #1401
• fix(engine): default shard_ids=None on clear_batches across all engines by @Adiactive in #1402
• docs: add community section with WeChat QR code to README 📢 by @mingcheng in #1409
• Supporting features for IcePop and KPop by @guojiapub in #1405
• fix: per-sample version tracking with loss_mask filter and multi-turn… by @pyq623 in #1408
• docs: add IcePop/KPop feature introduction by @guojiapub in #1424
• fix(vllm): forward frequency_penalty and stop in generation requests by @EazyReal in #1429
• fix(awex): allow disabling batch_send_recv use_group via AWEX_WU_USE_GROUP by @sitabulaixizawaluduo in #1414
• feat: disable megatron grad buffers CPU backup to save host memory by @HT-Yuan in #1393
• fix(network): find_free_ports ignores out-of-range exclude_ports by @EazyReal in #1436
• fix(reward): guard clevr_count_70k_reward_fn against scoring failures by @EazyReal in #1430
• feat(megatron): make MTP head opt-in to support Qwen3.6 MoE RL by @Adiactive in #1403
• feat(ppo): add CISPO loss surrogate (MiniMax-M1) by @EazyReal in #1412
• fix(CI): fix vlm_grpo CI OOM bug by @sitabulaixizawaluduo in #1438
• feat(cli): add experimental cli scaffold for service-style subcommands by @sitabulaixizawaluduo in #1440
• refactor: move 5 experimental modules into areal/v2 for 2.0 release by @sitabulaixizawaluduo in #1448
• feat: support v2 weight update disk mode for lora RL by @sitabulaixizawaluduo in #1450
• feat(cli): add training service cli by @sitabulaixizawaluduo in #1446
• feat(cli): add agent service cli by @sitabulaixizawaluduo in #1447
• feat(cli): add inference service cli by @sitabulaixizawaluduo in #1434
• feat(agent_service): add agent service with OpenClaw and Hermes examples by @IF007 in #1383
• feat(ppo): add reuse_train_logp proximal logp method by @Le8r0nJames in #1453
• fix(ppo): coerce ppo_n_minibatches to 1 for reuse_train_logp instead of raising by @Le8r0nJames in #1457
• Chore/add areal 2 report paper by @sitabulaixizawaluduo in #1461
• feat(launcher): add best-effort post-exit hook by @Le8r0nJames in #1459
• feat(megatron): add CP-safe vocab stats and MoE config support by @Le8r0nJames in #1460
• fix(ppo): handle variable-size trajectory groups in reward normalization by @Le8r0nJames in #1454
• fix(openai): parse tool_call arguments from JSON string to dict before chat template by @Le8r0nJames in #1463
• feat(swe): add SWE-bench RL training workflow example by @Le8r0nJames in #1462
• fix: fix safe-to-test CI workflow by @sitabulaixizawaluduo in #1456
• feat(openai): add proxy preprocessors and Qwen tool-call parsing by @Le8r0nJames in #1458
• test(ppo): update singleton leave-one-out group expectation by @Le8r0nJames in #1464
• chore: update AReaL 2.0 report paper by @sitabulaixizawaluduo in #1465
• chore: bump v2.0.0 by @sitabulaixizawaluduo in #1466

New Contributors

• @sebastiondev made their first contribution in #1323
• @mingcheng made their first contribution in #1348
• @haoyang9804 made their first contribution in #1346
• @jeis4wpi made their first contribution in #1352
• @zahrayousefijamarani made their first contribution in https://github.com/areal-proje...

v1.0.4

What's Changed

• fix: FSDP initialization for set-valued wrap class names by @Wangxiaoxiaoa in #1187
• chore: move figures into assets/figures/ and add community meeting folder by @garrett4wade in #1192
• feat(engine): lora support for MoE models (single node/ cross node) by @gursimar in #1159
• fix: handle integer device ids in ray rpc server by @Wangxiaoxiaoa in #1199
• chore: add @CormickKneey as maintainer and agent_service codeowner by @garrett4wade in #1201
• fix: serialize ray object refs in rpc payloads by @Wangxiaoxiaoa in #1198
• Fix:fix data service failures in IPv6-only environments with HTTP proxy by @TaoZex in #1208
• feat(service): add external model API support for inference service by @nuzant in #1183
• feat: add scaffolding rollout workflow by @WeiHaocheng in #1064
• feat(api): add unified RejectionSamplingConfig for async training by @guozhihao-224 in #1088
• feat(engine): Add Karmarkar-Karp algorithm by @TaoZex in #1151
• fix(infra): move data service seed to worker-level config by @garrett4wade in #1210
• docs(community): add first biweekly meeting record and news entry by @garrett4wade in #1215
• chore: update tencent meeting link by @garrett4wade in #1219
• chore: add new maintainers by @garrett4wade in #1220
• refactor(archon): extract inference bridge backends into sglang/vllm submodules by @garrett4wade in #1221
• gov: add maintainer and refine commit convention by @garrett4wade in #1227
• feat: add awex backend for weight update by @garrett4wade in #1214
• chore(deps): upgrade awex to 0.7.0 by @garrett4wade in #1228
• perf(trainer): reduce redundant offload/onload transitions in training loop by @yxs in #1163
• feat(eval): support for running eval before training by @gursimar in #1232
• gov: add new maintainer by @garrett4wade in #1234
• gov: add new maintainer by @garrett4wade in #1235
• feat: CP-local loss and configurable CUDA memory profiling by @yulangz in #1223
• feat: implement megatron weight update with awex by @ran-yan-hk in #1239
• feat: add disk-mode weight update flow to gateway by @garrett4wade in #1237
• feat(example): add Terminal Bench training example by @ActuallyEdward in #1224
• gov: update maintainer note by @garrett4wade in #1248
• feat: Implement Megatron PP and CP with Awex for Weight Update by @ran-yan-hk in #1246
• feat(trainer): add dpo by @HT-Yuan in #1190
• chore: add uv lock check in pre-commit hooks by @garrett4wade in #1259
• chore(deps): upgrade mbridge from 0.15.1 to 310e8fb by @Adiactive in #1258
• fix(docker): move venv out of /AReaL to avoid mount override by @sitabulaixizawaluduo in #1251
• feat: add configurable setup_timeout for data service gateway by @HT-Yuan in #1263
• test(infra): regression for single-controller SFT batch partitioning by @pratyush618 in #1255
• fix(engine): eagerly init HCCL subgroups to fix ref compute_logp on NPU by @pratyush618 in #1254
• feat(experimental): add MegatronEngine awex weight update adapter with EP by @ran-yan-hk in #1252
• refactor(experimental): rename WeightUpdateAdapter to AwexAdapter by @garrett4wade in #1269
• fix: teardown tcpstore race by @HT-Yuan in #1244
• refactor(experimental): reuse HTTP clients, add response models, and parallelize ops by @guozhihao-224 in #1253
• fix(engine): avoid duplicating multimodal tensors by @Wangxiaoxiaoa in #1272
• chore(experimental): suppress HTTP service logging to warning level by @garrett4wade in #1274
• fix: Update CI test GCP Image by @sitabulaixizawaluduo in #1277
• feat(experimental): add offload/onload endpoints for inference service by @garrett4wade in #1276
• feat(infra): add n_gpus_per_node abstract property to Scheduler API by @garrett4wade in #1275
• feat(engine): support direct engine construction via from_pretrained without config dataclass by @chenzhiyi021 in #1140
• feat: introduce Daytona cloud sandbox backend by @mu-hashmi in #1231
• fix(archon): support Qwen3.5 text checkpoints by @rchardx in #1278
• chore: update agenda for the next biweekly sync by @garrett4wade in #1285
• fix: apply_chat_template compatibility with transformers>=5.0 by @garrett4wade in #1280
• feat(engine): add Megatron support for Qwen2.5-VL by @Adiactive in #1281
• refactor(service): rename service controllers and unify service controller configs by @nuzant in #1265
• chore(dev): add /upgrade-deps skill by @garrett4wade in #1288
• fix(megatron): pass absolute total_train_steps as lr_decay_steps by @yulangz in #1289
• fix(megatron): compensate pipeline schedule's /num_microbatches in grad by @fishcrap in #1273
• perf(engine): avoid duplicating multimodal tensors in MegatronEngine by @Adiactive in #1291
• fix(infra): drain RTensor _fetch_buffer on all consumer workers by @guozhihao-224 in #1282
• fix(infra): gate RTensor.remotize on DP-head to plug _storage leak by @Adiactive in #1297
• feat(engine): add Qwen3-VL dense and MoE support to Megatron path by @Adiactive in #1301
• refactor(inference-service): remove unused release_capacity endpoint by @garrett4wade in #1293
• Add mbridge weight save as an option in Megatron by @hlyli in #1300
• chore(deps): upgrade runtime dependencies and CI workflow by @garrett4wade in #1206
• chore: upgrade GCP CI image by @garrett4wade in #1303
• fix(sft): report CP-invariant token-count stats (#1242) by @yulangz in #1249
• gov: add 0501 community meeting materials by @garrett4wade in #1306
• gov: enforce 2-approval merge policy on main by @garrett4wade in #1307
• chore: bump v1.0.4 by @garrett4wade in #1311

New Contributors

• @WeiHaocheng made their first contribution in #1064
• @yxs made their first contribution in #1163
• @ran-yan-hk made their first contribution in #1239
• @ActuallyEdward made their first contribution in #1224
• @sitabulaixizawaluduo made their first contribution in #1251
• @chenzhiyi021 made their first contribution in #1140
• @mu-hashmi made their first contribution in #1231

Full Changelog: v1.0.3...v1.0.4

v1.0.3

What's Changed

• chore(docker): add openclaw, ironclaw, zeroclaw, and nanobot-ai to runtime image by @garrett4wade in #1051
• feat(agent-service): add Agent Service microservice infrastructure by @CormickKneey in #1048
• feat(gateway): Add rollout gateway infrastructure with controller, router, and data proxy by @nuzant in #1043
• feat: estimators for kl divergence by @NicolasArias in #1060
• test(infra): speed up inference service integration tests by @nuzant in #1068
• fix(infra): simplify RTensor serialization in data proxy by @garrett4wade in #1067
• fix(rpc): resolve connection reset during RTensor fetch with large payloads by @pratyush618 in #1075
• docs: add gitcgr code graph badge by @vitali87 in #1073
• fix(openai): handle streaming responses in chat/completions endpoint by @Zijun9 in #1053
• fix: add PIL image and processor serialization for VLM RPC by @Adiactive in #1070
• refactor(api): migrate allocation_mode to per-engine backend fields by @garrett4wade in #1044
• chore(agents): add Codex harness and align AI workflows by @rchardx in #1082
• feat(platform): add NUMA CPU affinity binding for training engines by @HT-Yuan in #1083
• feat(commands): add fork workflow support to create-pr skill by @guozhihao-224 in #1092
• fix(rpc): batch HTTP RTensor fetches for large multimodal batches by @Wangxiaoxiaoa in #1077
• fix(fsdp): stabilize Qwen-VL rope-index argument binding and dtype by @Adiactive in #1094
• Refactor(vllm): use pause_generation from vllm instead of abort_all_req in areal_vllm_server by @HwVanICI in #1091
• feat: add BailingMoeV2.5 support with Lightning Attention + MLA + MoE + CP by @dingzhiqiang in #1079
• feat：support model training in IPv6-only environment by @TaoZex in #1072
• fix: fix pad_packed_tensor_dict by @HKAB in #1104
• feat: megatron bridge adaptation by @gursimar in #1056
• fix(engine): remove duplicate trust_remote_code kwarg in MegatronBridge init by @rchardx in #1107
• fix(dataloader): prevent data drop and padding during validation for accurate metrics by @Anguo-star in #1100
• fix(archon): add missing POST /data/batch endpoint to data proxy by @rchardx in #1105
• refactor(engine): abstract CUDA calls via current_platform in PerLayerOptimWrapper by @guozhihao-224 in #1108
• perf(fsdp): pipeline distributed weight sync with a single pending bucket by @HT-Yuan in #1074
• fix(engine): restore SGLang VLM training by @garrett4wade in #1098
• feat(archon): add FP8 blockwise training support by @rchardx in #1087
• chore(ci): update GCP CI image by @garrett4wade in #1115
• feat(inference-service): complete vLLM backend support in inference service by @garrett4wade in #1112
• fix(archon): harden FP8 blockwise training for TP and MoE scenarios by @rchardx in #1118
• feat(inference_service): add VLM image input support to OpenAI-compatible API by @garrett4wade in #1119
• feat(utils): add Trackio experiment tracking backend by @guozhihao-224 in #1113
• refactor(infra): decompose rpc_server into shared guard + blueprints by @garrett4wade in #1126
• refactor(agents): redesign review-pr taxonomy and sync flow by @rchardx in #1124
• feat(infra): add client-side fetch buffer for RTensor by @guozhihao-224 in #1122
• chore: fix gcp image to latest by @nuzant in #1130
• docs: add Trackio configuration to CLI reference by @rchardx in #1131
• feat(service): support online inference service by @nuzant in #1121
• fix(engine): for broken tree training due to bad indent in PR #1056 by @gursimar in #1135
• feat(service): add vllm backend support for inference service demo by @nuzant in #1136
• fix(api): add mode validation for WandBConfig and SwanlabConfig by @guozhihao-224 in #1134
• feat: enable LoRA RL-training in Megatron via megatron-bridge by @gursimar in #1123
• fix: harden padded distributed eval across training engines by @rchardx in #1109
• feat(ci): separate vllm and sglang pyproject.toml by @garrett4wade in #1141
• fix(vllm_ext): clear multimodal caches after generation pause by @Adiactive in #1144
• fix(ci): sync uv.vllm.lock with the current pyproject.vllm.toml by @garrett4wade in #1146
• fix(vllm_ext): XCCL lora weights update when PP>1 by buffering and merging PP shards by @gursimar in #1145
• chore: fix pre-commit by @garrett4wade in #1148
• Fix #1040: [Feature] Fixed bugs in Archon LoRA Backend by @JiwaniZakir in #1139
• feat(infra): add distributed data loading service by @garrett4wade in #1120
• refactor(infra): standardize list-first trajectory batch dispatch by @garrett4wade in #1150
• feat(infra): allow colocation with offloading and disk weight updates by @garrett4wade in #1157
• refactor: replace manual JSON parsing with Pydantic models by @koladefaj in #1154
• fix(engine): FSDP compute_logp fails for Qwen3.5 with dict attention_mask by @pratyush618 in #1153
• chore: update readme and enforce license by @garrett4wade in #1170
• chore: ensure SPDX license header in python source files by @garrett4wade in #1171
• fix: add missing pre-commit check file by @garrett4wade in #1173
• chore: add project governance for PyTorch ecosystem by @garrett4wade in #1174
• feat(infra): add microservice-based training service (controller v2) by @garrett4wade in #1169
• chore: renew qrcode by @garrett4wade in #1184
• feat(archon): support multi-node inference in gateway controller by @guozhihao-224 in #1178
• feat(agent-service): add Controller, Guard, and Claude Agent SDK example by @CormickKneey in #1177
• fix _update_weights_from_disk function to prevent training to be stuck by @asif07hossain in #1181
• refactor: mount data blueprint via WSGI and adopt Pydantic in engine blueprint by @koladefaj in #1179
• fix(engine): use meta device for non-rank-0 in FSDP memory_efficient_load by @yulangz in #1182
• ci: parallelize unit and integration tests across 4 GPU instances by @nuzant in #1185
• chore: dump v1.0.3 by @garrett4wade in #1191

New Contributors

• @pratyush618 made their first contribution in #1075
• @vitali87 made their first contribution in #1073
• @Adiactive made their first contribution in #1070
• @guozhihao-224 made their first contribution in #1092
• @TaoZex made their first contribution in #1072
• @HKAB made their first contribution in #1104
• @Anguo-star made their first contribution in #1100
• @JiwaniZakir made their first contribution in #1139
• @koladefaj made their first contribution in #1154
• @asif07hossain made their first contribution in #1181

Full Changelog: v1.0.2...v1.0.3

v1.0.2

Release Note

A massive thank you to our newest contributors who joined us for this release! The strength of this project lies in the collective expertise of the open-source community, and your work is what moves us forward.

🚀 Model & Architecture Updates

• Qwen3.5 Support: Added support for both dense and MoE (Mixture-of-Experts) variants of Qwen3.5 (archon backend, DP-only).

• On-Policy Distillation: Introduced native support for on-policy distillation.

• Added opt-in support for Hugging Face kernels and per-layer optimizer steps with a streaming H2D/D2H pipeline for FSDP.

🛠 Infrastructure & Scalability

• Docker & Runtime: Split Docker images into specialized sglang and vllm variants to allow different torch verions and fast docker image update.

📖 Documentation & Localization

• Bilingual Support: Launched comprehensive bilingual (EN/ZH) documentation, including a new translate-doc-zh command and fixed LaTeX rendering.

• New Guides: Added an online proxy mode training guide and revised existing online RL training tutorials.

What's Changed

• feat(models): shard vision encoder across Ulysses SP ranks by @aoshen524 in #929
• docs: fix broken documentation links by adding /en/ language prefix by @garrett4wade in #986
• [Feat] Add on-policy distillation support by @HwVanICI in #964
• fix: keep PPO token stats consistent under context parallelism by @yash27-lab in #990
• Upgrade GitHub Actions for Node 24 compatibility by @salmanmkc in #993
• Upgrade GitHub Actions to latest versions by @salmanmkc in #994
• chore: add Python 3.11 support (requires-python >=3.11,<3.13) by @garrett4wade in #991
• chore(ci): replace format-check with pre-commit CI and add commit-msg hooks by @rchardx in #998
• feat(docs): add bilingual documentation with translate-doc-zh command by @ZiyiTsang in #995
• refactor: flatten sub-module imports to use parent package re-exports by @NJX-njx in #996
• feat(infra): split Docker image into sglang and vllm variants by @garrett4wade in #985
• docs: fix broken LaTeX rendering in bilingual docs by @rchardx in #1004
• fix: CPU-only support on macOS by @zhanghaotong in #1003
• feat(archon): add moe_router_dtype config for FP32 router gate GEMM by @rchardx in #1009
• refactor: replace string literals with enums and fix logging issues- … by @HT-Yuan in #1008
• feat(fsdp): add per-layer optimizer step with streaming H2D/D2H pipeline by @aoshen524 in #983
• Update NPU doc for v1.0.1 release by @HwVanICI in #1022
• feat(archon): add Qwen3.5 dense and MoE model support (DP-only) by @rchardx in #1012
• fix: LoRA and XCCL openai_serving_models LoRA versioning bug by @TinLongYu in #1021
• fix: unify RPC error response JSON key to "error" across server and s… by @HT-Yuan in #1019
• fix: close sockets on bind failure, fix exit traceback in trainers by @mango766 in #1032
• fix(vllm): harden runtime LoRA alias handling for XCCL updates by @Wangxiaoxiaoa in #1039
• fix(ci): minor fixing ruff format by @garrett4wade in #1041
• docs(github): improve PR template checklist and type-of-change section by @garrett4wade in #1042
• docs: add online proxy mode training guide by @Zijun9 in #1006
• Add opt-in support for Hugging Face kernels by @lewtun in #1033
• fix(archon): Wrap router gate in nn.Module for DTensor hook compatibility by @fishcrap in #1029
• chore(deps): bump sglang, vllm, megatron-core and restructure Dockerfile by @garrett4wade in #1010
• ci(infra): add GCP image baking workflow for CI acceleration by @garrett4wade in #1045
• refactor(infra): simplify RTensor to single-shard and adopt per-trajectory list pipeline by @fishcrap in #1017
• ci(infra): add GCP image baking workflow and update CI image by @garrett4wade in #1047
• docs: revise online RL training tutorial for EN and ZH by @garrett4wade in #1049
• chore: prepare v1.0.2 release by @garrett4wade in #1050

New Contributors

• @aoshen524 made their first contribution in #929
• @yash27-lab made their first contribution in #990
• @salmanmkc made their first contribution in #993
• @NJX-njx made their first contribution in #996
• @zhanghaotong made their first contribution in #1003
• @HT-Yuan made their first contribution in #1008
• @mango766 made their first contribution in #1032
• @Wangxiaoxiaoa made their first contribution in #1039
• @Zijun9 made their first contribution in #1006
• @lewtun made their first contribution in #1033

Full Changelog: v1.0.1...v1.0.2

v1.0.1

Release Note

A patch release that fixes a dependency issue in the docker image and enriches the documentation and testing of the OpenClaw example.

What's Changed

• fix(config): Fix openclaw config typo and increase max_tokens_per_mb by @fishcrap in #959
• docs(openclaw): Replace hardcoded admin key with placeholder in README by @fishcrap in #967
• feat: Fully Support MIS/TIS to stablizing rollout-training mismatch by @ZiyiTsang in #930
• refactor(api): move validation into config post_init methods by @rchardx in #970
• fix(openai-proxy): return None for empty trajectory in online mode by @fishcrap in #971
• Ray placement group refactor and preliminary architecture for multinode inference instances by @hlyli in #966
• feat:Add chinese doc by @ZiyiTsang in #969
• fix(api): replace Literal type with str for SchedulingSpec.ray_placement_strategy by @garrett4wade in #976
• update readme by @xssstory in #974
• test(examples): Add OpenClaw online RL integration test by @fishcrap in #977
• fix: pinning torchao version to 0.15.0 by @garrett4wade in #981
• bump v1.0.1 by @garrett4wade in #982

New Contributors

• @hlyli made their first contribution in #966

Full Changelog: v1.0.0...v1.0.1

v1.0.0

🚀 Key Highlights

Release Notes

Online RL Training

• Seamlessly train any agents by configuring a base_url and api_key—no code changes required and no heavy dependencies.
• Check out the OpenClaw RL training example for more details.

Archon Engine

• A fully working, PyTorch-native 5D parallel training engine.
• Includes features like:
  • Automatic HF format conversion
  • Zero-bubble pipelining
  • torch.compile
  • FSDP (Fully Sharded Data Parallel)
  • Selective activation support

AI-Assisted Coding

• Official commands and skills to streamline development and enable easy customization.

Infrastructure Upgrade

• Transition from the previous SPMD architecture to a more efficient single-controller architecture.

uv Installation Support

• Easily set up training environments by running the simple command: uv sync.

What's Changed

• feat: replace legacy math parsing with math-verify by @rchardx in #739
• Add installation instructions for Ascend NPU by @HwVanICI in #748
• [Bug Fix] Fix Tools compatibility, max_token restrictions, and EOS token issues in Proxy mode by @yulangz in #736
• refactor: modify engine and controllers to support the single-controller mode with the same trainer by @garrett4wade in #753
• VLM Training on NPU by @HwVanICI in #746
• refactor: move device utilities to platform classes and io_struct by @garrett4wade in #757
• Fix: Implement get_device_stats() for train_controller by @HwVanICI in #762
• refactor: single-source task_id generation in submit methods by @garrett4wade in #759
• [Bug Fix] Camel example with wrong and missing agent arguments by @HwVanICI in #766
• feat: use name_resolve for worker discovery and fix perf_tracer in the single-controller mode by @garrett4wade in #764
• [Feature] Implement Single-Controller XCCL Weight Update by @HwVanICI in #754
• feat: Implement slurm scheduler by @garrett4wade in #767
• Ray Scheduler Implementation for Single Controller by @HwVanICI in #741
• refactor: use callbacks to implement xccl weigh transfer and avoid busy waiting during rollout by @garrett4wade in #769
• [Testing] Update GCP image to accelerate CI testing by @nuzant in #772
• refactor: unifying launcher, scheduling spec, yaml configs, and training scripts by @garrett4wade in #770
• feat: improve logging by @garrett4wade in #771
• refactor: separate megatron imports and installation from FSDP by @garrett4wade in #773
• minor fix: vLLM LoRA request cleanup for issue #751 by @TinLongYu in #765
• fix: refactoring proximal logp recompute condition by @garrett4wade in #780
• [Feat] Add FP8 training support by @fishcrap in #758
• Enhance host IP detection in areal.utils.network by @HwVanICI in #778
• chore: remove the ad-hoc should_broadcast parameter in rpc servers by @garrett4wade in #774
• feat: support colocated engines in the single-controller mode by @garrett4wade in #779
• chore: update readme by @garrett4wade in #782
• docs: restructure AGENTS.md and add CLAUDE.md symlink by @rchardx in #783
• chore: Expose error when lauch sglang server by @ZiyiTsang in #781
• refactor: simplifying the implementation of customized workflow with context management by @garrett4wade in #785
• chore: Expose error when launching vllm server by @garrett4wade in #790
• refactor: allow dynamic batch size without the dynamic_filtering function by @garrett4wade in #786
• fix: fix ray scheduler in the single-controller mode by @garrett4wade in #791
• fix inference engine addr resolving logic by @garrett4wade in #792
• chore: minor fix doc formula by @ZiyiTsang in #793
• doc: update docs for grpo and related algorithms by @garrett4wade in #794
• refactor: migrate grouped rollout from customized workflows to inference engines by @garrett4wade in #789
• Single-controller LoRA RL fine-tuning with vLLM by @gursimar in #735
• [Feature] Group-level data redistribution by @nuzant in #800
• critical fix: passing is_eval and group_size from rollout controller to engines by @garrett4wade in #801
• Update NPU doc by @HwVanICI in #803
• feat: add Archon Engine - PyTorch native FSDP2 training backend by @rchardx in #799
• [Feature] Tree training support (Megatron Engine) for agentic RL training by @nuzant in #804
• Add NPU RLVR example by @HwVanICI in #798
• [Bug Fix] XCCL weight synchronization fix for the single controller lora by @gursimar in #796
• [Bug Fix] Fix import error introduced by tree training PR by @nuzant in #808
• chore: remove legacy code, config, and documentation by @garrett4wade in #806
• fix: update tree_attn function name to patch_bridge_for_tree_training by @rchardx in #809
• fix: prevent fake PID killing in LocalScheduler tests by @rchardx in #810
• feat(archon): add torch.compile support and profiling tools by @rchardx in #807
• Add RayScheduler to sft.py by @HwVanICI in #814
• fix: add lm_head.weight into index when index file exists by @jwhj in #816
• feat: use subprocess to fork colocated workers by @garrett4wade in #815
• feat(archon): add Context Parallelism (Ulysses SP) support by @rchardx in #817
• refactor(data): simplify pad_mb_list alignment parameters by @rchardx in #820
• refactor: unify HTTP client management in workflow_context by @garrett4wade in #819
• feat(archon): enable TP + AC + compile compatibility with _WaitAsyncWrapper by @rchardx in #821
• [FEAT] Add direct TE FP8-PyTorch FP8 conversion by @fishcrap in #802
• refactor(core): simplify HTTP client lifecycle with event loop cleanup by @garrett4wade in #823
• feat: Add AgentWorkflow API and migrate workflow resolution to RemoteInfEngine by @garrett4wade in #825
• feat(scheduler): refactor fork_workers to public API with custom command support by @garrett4wade in #826
• [FIX] correct vLLM config defaults for chunked prefill and prefix caching by @fishcrap in #827
• refactor(openai): modularize proxy architecture and add inline mode by @garrett4wade in #829
• chore(doc): update readme by @garrett4wade in #830
• fix(test): Fix math-verify tests by @garrett4wade in #831
• feat(archon): add Expert Parallelism (EP) support for MoE models by @rchardx in #833
• fix(moe): correct histc max param by @rchardx in #835
• feat(archon): add explicit FSDP prefetching for EP by @rchardx in #834
• feat(archon): add EP-aware padding wrapper for MoE grouped_mm by @rchardx in #836
• testing: fix CI, skip tests that cannot run on A100 GPUs by @nuzant in #838
• feat(archon): add Expert Tensor Parallelism (ETP) support for MoE models by @rchardx in #839
• feat: support tree training for FSDP engine by @nuzant in #837
• fix: remove duplicate setup in gsm8k_rl by @v3nividiv1ci in #842
• refactor(archon): cleanup parallel dims and FSDP config for pipeline parallelism by @rchardx in #841
• refactor(tree_attn): decouple FSDP and Megatron implementations by @rchardx in https://github.com/inclusionAI/AReaL/pu...

v1.0.0.rc1

Pre-release for 1.0.0.

v0.5.3

Highlights

This is a patch release primarily for delivering the latest docker image for testing.

We will include well-documented features in the next major release.

v0.5.2

Highlights

This is a patch release primarily for delivering the latest docker image with torch 2.9.1, vllm 0.14.0, and sglang 0.5.7 supports.

We will include well-documented features in the next major release.

v0.5.1

Highlights

This is a patched release upon v0.5.0.

• A new docker image with math-verify and the latest ruff.
• Support for PPO critic model support with Megatron engine.
• Refactored FSDP/Megatron engine implementations.
• Implement efficient RPC tensor transfer with RTensor (aka the original DistributedBatch).
• Beam seach support for vLLM.

What's Changed

• fix: change checkpoint cleanup flag to fix update_weights_from_disk in single-controller mode by @HwVanICI in #711
• fix: prevent port overflow in vLLM server with high data parallelism (fixes #652) by @HsiaoTsan in #653
• refactor: refactor train engine high level APIs by @aaaandychen in #658
• [Fix] Fix the bug that experiments cannot properly exit in the TIR example by @nuzant in #712
• chore: print more information in concat mode and handle empty tool calls for easy debugging by @nuzant in #713
• chore: trim tests in CI by @garrett4wade in #714
• refactor: enforce task_id creation, access, and manipulation in inference engines by @garrett4wade in #715
• refactor: redesign TrainEngine API with cleaner abstractions by @rchardx in #719
• [Testing] Add SFT/GRPO integration test for Megatron train engine. by @nuzant in #726
• [FEAT] VLLM support for VLM training by @HwVanICI in #698
• feat: Support beam_search in vllm backend by @ZiyiTsang in #721
• fix: update multi-turn math test configuration by @rchardx in #727
• fix: fix logic error in beam search support check by @rchardx in #728
• feat: add PPO Critic model support for MegatronEngine by @rchardx in #729
• feat: implement RTensor for metadata transfer in the single-controller mode by @garrett4wade in #731
• fix: fix multi-turn proxy example by @dhh1995 in #733
• minor fix: fix openai cache test, add it in CI test suite, and remove OOD todos/fixmes in Megatron engine by @garrett4wade in #732
• [Feat] XCCL-updates for single LoRA functionality for ascend-vLLM by @gursimar in #679
• fix: use group_size=1 for eval in proxy examples by @dhh1995 in #737
• feat: add ignore_eos and skip_special_tokens generation params by @rchardx in #738
• chore: update datasets to version 3.0.0 or higher for inner API compatibility by @ZiyiTsang in #720
• feat: build the docker image with math-verify and the latest ruff by @garrett4wade in #744
• bump v0.5.1 by @garrett4wade in #745

New Contributors

• @HsiaoTsan made their first contribution in #653
• @aaaandychen made their first contribution in #658
• @gursimar made their first contribution in #679

Full Changelog: v0.5.0...v0.5.1