Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Agentic Pipeline Architecture Diagram

![Agentic Pipeline Architecture Diagram](../../../static/img/AgenticPipeline.jpg)
![Agentic Pipeline Architecture Diagram](../../static/img/AgenticPipeline.jpg)

## AgenticPipeline

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Pipeline Config

Refer to [RLVR Pipeline Start](https://alibaba.github.io/ROLL/docs/English/StepByStep/rlvr_pipeline_start) and [Agentic Pipeline Start](https://alibaba.github.io/ROLL/docs/English/StepByStep/agent_pipeline_start) for more details about RLVR/Agentic pipeline configurations and reward settings.
Refer to [RLVR Pipeline Start](https://alibaba.github.io/ROLL/docs/StepByStep/rlvr_pipeline_start) and [Agentic Pipeline Start](https://alibaba.github.io/ROLL/docs/StepByStep/agent_pipeline_start) for more details about RLVR/Agentic pipeline configurations and reward settings.

```yaml
rollout_batch_size: 64
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ To better understand ROLL's configuration system, the UML diagram of RLVRConfig
- In WorkerConfig, ModelArguments, TrainingArguments, GeneratingArguments, DataArguments, and backend StrategyArguments held by the role are defined as needed
- WorkerConfig can be extended as needed to construct new application configurations

![RLVRConfig Diagram](../../../static/img/roll_config_diagram.png)
![RLVRConfig Diagram](../../static/img/roll_config_diagram.png)

## Mapping between YAML Configuration and PipelineConfig

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## 🐳 Install from Docker

We provide pre-built Docker images both on CUDA and ROCm for a quick start. Choose your desired image from the [Image Addresses](https://alibaba.github.io/ROLL/docs/English/QuickStart/image_address).
We provide pre-built Docker images both on CUDA and ROCm for a quick start. Choose your desired image from the [Image Addresses](https://alibaba.github.io/ROLL/docs/QuickStart/image_address).

## 🛠️ Install from Custom Environment

Expand Down Expand Up @@ -35,7 +35,7 @@ git clone https://github.com/alibaba/ROLL.git
cd ROLL
pip install -r requirements.txt # Or follow your specific installation steps
```
We highly suggest to use pre-built Docker images from [Image Addresses](https://alibaba.github.io/ROLL/docs/English/QuickStart/image_address) instead of installation from Custom Environment for ROCm users.
We highly suggest to use pre-built Docker images from [Image Addresses](https://alibaba.github.io/ROLL/docs/QuickStart/image_address) instead of installation from Custom Environment for ROCm users.

## 🔄 About Model Checkpoint Format

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ curl -fsSL https://github.com/alibaba/ROLL/blob/main/scripts/install_docker_nvid
```

## Environment Configuration
Choose your desired Docker image from the [image addresses](https://alibaba.github.io/ROLL/docs/English/QuickStart/image_address). The following example uses *torch2.6.0 + vLLM0.8.4*
Choose your desired Docker image from the [image addresses](https://alibaba.github.io/ROLL/docs/QuickStart/image_address). The following example uses *torch2.6.0 + vLLM0.8.4*
```shell
# 1. Start a Docker container with GPU support, expose container ports, and keep the container running
sudo docker run -dit \
Expand Down Expand Up @@ -56,7 +56,7 @@ Notes:
bash examples/agentic_demo/run_agentic_pipeline_frozen_lake_multi_nodes_demo.sh
```
After the Ray cluster starts, you will see log examples like the following:
![log_ray_multi_nodes](../../../static/img/log_ray_multi_nodes.png)
![log_ray_multi_nodes](../../static/img/log_ray_multi_nodes.png)

3. Configure environment variables on the worker node
```shell
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ Check the total number of GPUs requested and the `device_mapping` configuration.

When ROLL distributes data, it will distribute `rollout_batch_size` samples to each `actor_train` worker according to DP size, and then calculate the samples for each gradient update according to `gradient_accumulation_steps`. The configuration results in 0 when divided.

For detailed configuration logic, refer to the manual: [Training Arguments](https://alibaba.github.io/ROLL/docs/English/QuickStart/config_guide#training-arguments-training_args)
For detailed configuration logic, refer to the manual: [Training Arguments](https://alibaba.github.io/ROLL/docs/QuickStart/config_guide#training-arguments-training_args)

### Error: `AssertionError: batch_size 32 < chunks 64`

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ curl -fsSL https://github.com/alibaba/ROLL/blob/main/scripts/install_docker_nvid
```

## Environment Configuration
Choose your desired Docker image from the [image addresses](https://alibaba.github.io/ROLL/docs/English/QuickStart/image_address). The following example uses *torch2.6.0 + vLLM0.8.4*
Choose your desired Docker image from the [image addresses](https://alibaba.github.io/ROLL/docs/QuickStart/image_address). The following example uses *torch2.6.0 + vLLM0.8.4*
```shell
# 1. Start a Docker container with GPU support, expose container ports, and keep the container running
sudo docker run -dit \
Expand Down Expand Up @@ -41,11 +41,11 @@ bash examples/agentic_demo/run_agentic_pipeline_frozen_lake_single_node_demo.sh
```

Example log screenshots during pipeline execution:
![log_pipeline_start](../../../static/img/log_pipeline_start.png)
![log_pipeline_start](../../static/img/log_pipeline_start.png)

![log_pipeline_in_training](../../../static/img/log_pipeline_in_training.png)
![log_pipeline_in_training](../../static/img/log_pipeline_in_training.png)

![log_pipeline_complete](../../../static/img/log_pipeline_complete.png)
![log_pipeline_complete](../../static/img/log_pipeline_complete.png)

## Reference: Single V100 GPU Memory Configuration Key Points
```yaml
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -138,15 +138,15 @@ dapo_17k_with_python_code:
}
```

![math dapo + python_tool](../../../../static/img/math_python_tool.png)
![math dapo + python_tool](../../../static/img/math_python_tool.png)

### QA + SearchTool

#### Training Configuration
- HotpotQA: `examples/qwen3_agentic_gem/gem_math_hotpotqa.yaml`
- HotpotQA + Search: `examples/qwen3_agentic_gem/gem_math_hotpotqa_search.yaml`

![qa + search](../../../../static/img/qa_search.png)
![qa + search](../../../static/img/qa_search.png)

#### Search Engine Startup Method

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -298,4 +298,4 @@ rollout_dump_dir: odps://odps_project/tables/table_name/ds=${model_name}

A: You can debug the trajectory generation process by configuring log levels and adding custom logs. Trajectory data will be completely saved in JSON format for easy analysis.

For multi-round interaction local debugging, refer to the documentation: [Debug Guide](../../../简体中文/快速开始/debug_guide.md)
For multi-round interaction local debugging, refer to the documentation: [Debug Guide](../../QuickStart/debug_guide.md)
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,7 @@ actor_infer:
- Configuration example: `examples/qwen2.5-7B-rlvr_megatron/rlvr_config_async.yaml`

### Off-Policy Algorithms
ROLL also supports various Off-Policy algorithms. For detailed information, please refer to: `docs_roll/docs/English/UserGuide/algorithms/offpolicy_setting.md`
ROLL also supports various Off-Policy algorithms. For detailed information, please refer to: `docs_roll/docs/UserGuide/algorithms/offpolicy_setting.md`

Configuration example: `examples/qwen2.5-7B-rlvr-offpolicy/rlvr_config.yaml`

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,7 @@ python examples/start_agentic_pipeline.py \

### Step 5: Output and Results

* **Trained Model** – Checkpoints are saved in `checkpoint_config`, refer to documentation [checkpoint_and_resume](././checkpoint_and_resume.md) for details.
* **Trained Model** – Checkpoints are saved in `checkpoint_config`, refer to documentation [checkpoint_and_resume](../../UserGuide/checkpoint_and_resume) for details.
* **Evaluation Metrics** – Recorded in TensorBoard and terminal.
* **Rendered Frames** – If `render_save_dir` is configured, environment rendered frames will be saved in that directory, facilitating visualization of the interaction process.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,15 +42,15 @@ Leveraging a multi-role distributed architecture with Ray for flexible resource
[Documents](https://alibaba.github.io/ROLL/)

### Quick Start
[Installation](https://alibaba.github.io/ROLL/docs/English/QuickStart/installation)
[Config System Explanation](https://alibaba.github.io/ROLL/docs/English/QuickStart/config_system)
[Debugging Guide](https://alibaba.github.io/ROLL/docs/English/QuickStart/debugging_guide_en)
[Trackers and Metrics](https://alibaba.github.io/ROLL/docs/English/UserGuide/trackers_and_metrics)
[Checkpoint Saving and Resuming Guide](https://alibaba.github.io/ROLL/docs/English/UserGuide/checkpoint_and_resume)
[Converting MCoreAdapter Models to Hugging Face Format](https://alibaba.github.io/ROLL/docs/English/UserGuide/megatron_convert_2_hf)
[Quick Start: Single-Node Deployment Guide](https://alibaba.github.io/ROLL/docs/English/QuickStart/single_node_quick_start)
[Quick Start: Multi-Node Deployment Guide](https://alibaba.github.io/ROLL/docs/English/QuickStart/multi_node_quick_start)
[Frequently Asked Questions](https://alibaba.github.io/ROLL/docs/English/QuickStart/qa_issues)
[Installation](https://alibaba.github.io/ROLL/docs/QuickStart/installation)
[Config System Explanation](https://alibaba.github.io/ROLL/docs/QuickStart/config_system)
[Debugging Guide](https://alibaba.github.io/ROLL/docs/QuickStart/debugging_guide_en)
[Trackers and Metrics](https://alibaba.github.io/ROLL/docs/UserGuide/trackers_and_metrics)
[Checkpoint Saving and Resuming Guide](https://alibaba.github.io/ROLL/docs/UserGuide/checkpoint_and_resume)
[Converting MCoreAdapter Models to Hugging Face Format](https://alibaba.github.io/ROLL/docs/UserGuide/megatron_convert_2_hf)
[Quick Start: Single-Node Deployment Guide](https://alibaba.github.io/ROLL/docs/QuickStart/single_node_quick_start)
[Quick Start: Multi-Node Deployment Guide](https://alibaba.github.io/ROLL/docs/QuickStart/multi_node_quick_start)
[Frequently Asked Questions](https://alibaba.github.io/ROLL/docs/QuickStart/qa_issues)

### UserGuide

Expand Down
Empty file removed docs_roll/docs/index.md
Empty file.
16 changes: 13 additions & 3 deletions docs_roll/docusaurus.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,17 @@ const config = {
// to replace "en" with "zh-Hans".
i18n: {
defaultLocale: 'en',
locales: ['en'],
locales: ['en', 'zh-Hans'],
localeConfigs: {
en: {
label: 'English',
direction: 'ltr',
},
'zh-CN': {
label: '简体中文',
direction: 'ltr',
},
},
},

presets: [
Expand Down Expand Up @@ -115,11 +125,11 @@ const config = {
items: [
{
label: 'ROLL单机实践手册',
to: '/docs/简体中文/快速开始/single_node_quick_start_cn',
to: '/docs/QuickStart/single_node_quick_start',
},
{
label: '配置指南',
to: '/docs/简体中文/快速开始/config_guide_cn',
to: '/docs/QuickStart/config_guide',
},
],
},
Expand Down
Loading