Make long-document SDG docs paths portable

Author: intern_nem_dev_1

src/nemotron/recipes/data/sdg/long-document/README.md

@@ -99,7 +99,7 @@ The same stages, dispatched via `nemo-run`. Add `--run <profile>` (attached) or
 ```bash
 nemotron data sdg long-document seed --batch prep \
     -c 01-seed num_docs=50 \
-    output_dir=/lustre/.../sdg/seeds
+    output_dir=${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/seeds
 ```
 
 For producer stages and the judge, you have two options for the model endpoint:
@@ -112,9 +112,9 @@ Same flow as standalone, plus `--run <profile>` / `--batch <profile>`. The recip
 nemotron data sdg long-document ocr --batch prep \
     -c 02-ocr \
     vllm_endpoint=http://compute-node-0001:8000/v1 \
-    seed_path=/lustre/.../seeds/seed_per_page.parquet \
+    seed_path=${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/seeds/seed_per_page.parquet \
     num_records=100 \
-    artifact_path=/lustre/.../sdg/ocr
+    artifact_path=${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/ocr
 ```
 
 #### Option B — let the CLI auto-deploy with `--serve`
@@ -134,15 +134,15 @@ Drop the `vllm_endpoint=…` argument when using `--serve` — it's set automati
 # runs the recipe against it, tears the deployment down on exit.
 nemotron data sdg long-document ocr --batch prep --serve \
     -c 02-ocr \
-    seed_path=/lustre/.../seeds/seed_per_page.parquet \
+    seed_path=${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/seeds/seed_per_page.parquet \
     num_records=100 \
-    artifact_path=/lustre/.../sdg/ocr
+    artifact_path=${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/ocr
 
 # Override the default deployment (e.g. test a different vLLM tuning):
 nemotron data sdg long-document single-page-qa --batch prep --serve \
     --serve-config qwen3-vl-30b \
     -c 06-single-page-qa \
-    seed_path=/lustre/.../seeds/seed_per_page.parquet \
+    seed_path=${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/seeds/seed_per_page.parquet \
     num_records=10000
 ```
 
@@ -170,9 +170,9 @@ nemotron data sdg long-document ocr  --batch prep --serve -c 02-ocr ...
 
 The `--serve` *serve* task always lands on a GPU partition (the cluster's `sdg_serve_partition` in `env.toml`, defaulting to `interactive`); the *client* task uses the regular `run_partition` / `batch_partition` of the env profile.
 
-### Lustre paths
+### Shared output paths
 
-- `seed_path`, `output_dir`, `artifact_path` should all be absolute paths on shared storage (Lustre on dlw). Relative paths resolve to the slurm job's working directory, which is generally not what you want.
+- `seed_path`, `output_dir`, `artifact_path` should all point at durable shared output storage. The examples below use `${NEMO_RUN_DIR:-.}/output/data/sdg/long-document`; set `NEMO_RUN_DIR` to a path visible to the client job and any manually launched model server for your cluster.
 - The serve sentinel lives at `${remote_job_dir}/sdg-deploy/<recipe-name>/<timestamp>/endpoint`.
 
 ### HF / API tokens
@@ -240,13 +240,13 @@ Slurm + Pyxis (1× H100):
 srun --partition=interactive --nodes=1 --ntasks=1 --gres=gpu:1 \
      --time=04:00:00 \
      --container-image=vllm/vllm-openai:v0.14.1 \
-     --container-mounts=/lustre:/lustre \
+     --container-mounts=${NEMO_RUN_DIR:-.}:${NEMO_RUN_DIR:-.} \
      bash -c "pip install open-clip-torch albumentations timm && \
               vllm serve nvidia/NVIDIA-Nemotron-Parse-v1.1 \
                 --tensor-parallel-size 1 \
                 --max-model-len 9000 \
                 --gpu-memory-utilization 0.85 \
-                --chat-template /lustre/.../chat_template.jinja \
+                --chat-template ${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/chat_template.jinja \
                 --trust-remote-code"
 ```
 
@@ -342,31 +342,31 @@ uv run --no-project 09-frontier-judge-sdg.py \
 # 1. Seed (CPU partition).
 nemotron data sdg long-document seed --batch prep \
     -c 01-seed \
-    output_dir=/lustre/.../sdg/seeds \
+    output_dir=${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/seeds \
     num_docs=1000
 
 # 2. OCR — auto-deploys nemotron-parse on a GPU node.
 nemotron data sdg long-document ocr --batch prep --serve \
     -c 02-ocr \
-    seed_path=/lustre/.../sdg/seeds/seed_per_page.parquet \
+    seed_path=${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/seeds/seed_per_page.parquet \
     num_records=10000 \
-    artifact_path=/lustre/.../sdg/ocr
+    artifact_path=${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/ocr
 
 # 3. Single-page QA — auto-deploys Qwen3-VL-235B on 4× GPUs.
 nemotron data sdg long-document single-page-qa --batch prep --serve \
     -c 06-single-page-qa \
-    seed_path=/lustre/.../sdg/seeds/seed_per_page.parquet \
+    seed_path=${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/seeds/seed_per_page.parquet \
     num_records=100000 \
-    artifact_path=/lustre/.../sdg/single_page_qa
+    artifact_path=${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/single_page_qa
 
 # 4. Judge.
 nemotron data sdg long-document judge --batch prep \
     -c 09-judge \
-    seed_path=/lustre/.../sdg/single_page_qa/.../generated.parquet \
+    seed_path=${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/single_page_qa/.../generated.parquet \
     endpoint=https://generativelanguage.googleapis.com/v1beta \
     model_id=gemini-2.5-flash-preview-04-17 \
     api_key_env=GEMINI_API_KEY \
-    artifact_path=/lustre/.../sdg/judged_single_page_qa
+    artifact_path=${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/judged_single_page_qa
 ```
 
 ## Troubleshooting
@@ -418,16 +418,17 @@ Recommended contents for `./published_dataset`:
 ### Private path: internal storage plus artifact registration
 
 ```bash
-mkdir -p /lustre/team/datasets/long-document-understanding-sdg/v1
-cp -R ./published_dataset/. /lustre/team/datasets/long-document-understanding-sdg/v1/
+export LONG_DOC_PUBLISH_DIR="${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/internal/long-document-understanding-sdg/v1"
+mkdir -p "$LONG_DOC_PUBLISH_DIR"
+cp -R ./published_dataset/. "$LONG_DOC_PUBLISH_DIR"/
 ```
 
 If your environment uses Nemotron artifact logging:
 
 ```bash
 nemotron kit log-artifact data \
     --name omni3-long-document-sdg \
-    --path /lustre/team/datasets/long-document-understanding-sdg/v1
+    --path "$LONG_DOC_PUBLISH_DIR"
 ```
 
 Otherwise register manually through your standard W&B flow with `wandb.log_artifact`.

src/nemotron/recipes/data/sdg/long-document/config/02-ocr.yaml

@@ -3,7 +3,7 @@
 # `vllm_endpoint` and `seed_path` are required; supply them on the CLI:
 #   nemotron data sdg long-document ocr --run dlw -c 02-ocr \
 #       vllm_endpoint=http://node:8000/v1 \
-#       seed_path=/lustre/.../seed_per_page.parquet \
+#       seed_path=${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/seeds/seed_per_page.parquet \
 #       num_records=100
 
 vllm_endpoint: null

tests/recipes/data/test_sdg_long_document_docs_portability.py

@@ -0,0 +1,64 @@
+from pathlib import Path
+
+REPO_ROOT = Path(__file__).resolve().parents[3]
+LONG_DOCUMENT_DIR = REPO_ROOT / "src/nemotron/recipes/data/sdg/long-document"
+README = LONG_DOCUMENT_DIR / "README.md"
+OCR_CONFIG = LONG_DOCUMENT_DIR / "config/02-ocr.yaml"
+PORTABLE_ROOT = "${NEMO_RUN_DIR:-.}/output/data/sdg/long-document"
+
+
+def _read(path: Path) -> str:
+    return path.read_text(encoding="utf-8")
+
+
+def test_long_document_docs_do_not_use_developer_lustre_paths() -> None:
+    for path in (README, OCR_CONFIG):
+        text = _read(path)
+        assert "/lustre/" not in text, f"{path} still contains a concrete /lustre/ example"
+
+
+def test_long_document_docs_use_portable_output_examples() -> None:
+    readme_text = _read(README)
+    ocr_config_text = _read(OCR_CONFIG)
+
+    for suffix in (
+        "seeds",
+        "seeds/seed_per_page.parquet",
+        "ocr",
+        "single_page_qa",
+        "judged_single_page_qa",
+        "chat_template.jinja",
+        "internal/long-document-understanding-sdg/v1",
+    ):
+        assert f"{PORTABLE_ROOT}/{suffix}" in readme_text
+
+    assert f"{PORTABLE_ROOT}/seeds/seed_per_page.parquet" in ocr_config_text
+
+
+def test_long_document_docs_preserve_cli_commands_and_options() -> None:
+    readme_text = _read(README)
+    ocr_config_text = _read(OCR_CONFIG)
+
+    for snippet in (
+        "nemotron data sdg long-document seed",
+        "nemotron data sdg long-document ocr",
+        "nemotron data sdg long-document single-page-qa",
+        "nemotron data sdg long-document judge",
+        "--batch prep",
+        "--serve",
+        "--serve-config",
+        "--config config/02-ocr.yaml",
+        "vllm_endpoint=",
+        "seed_path=",
+        "artifact_path=",
+        "output_dir=",
+    ):
+        assert snippet in readme_text
+
+    for snippet in (
+        "nemotron data sdg long-document ocr --run dlw -c 02-ocr",
+        "vllm_endpoint=http://node:8000/v1",
+        "seed_path=",
+        "num_records=100",
+    ):
+        assert snippet in ocr_config_text

workspace/interns/intern_nem_dev_1/status.md

@@ -1,12 +1,12 @@
 # intern_nem_dev_1 - 状态
 
-<!-- METADATA:STATUS=Working,TASK=task154_super3_rl_data_prep_doc_output_portability_s1,ROLE=independent,SESSION=1 -->
+<!-- METADATA:STATUS=Working,TASK=task157_data_sdg_long_document_docs_path_portability_s1,ROLE=dev,SESSION=1 -->
 
 | 字段 | 值 |
 |------|-----|
 | Name | intern_nem_dev_1 |
 | Status | Working |
-| Current Task | task154_super3_rl_data_prep_doc_output_portability_s1 |
-| PR | https://github.com/songCNMS/Nemotron/pull/261 |
+| Current Task | task157_data_sdg_long_document_docs_path_portability_s1 |
+| PR | pending |
 | Session | 1 |
-| Recent Progress | Opened PR #261 for task154; Super3 RL data-prep doc M0 output example now uses NEMO_RUN_DIR-relative path and focused docs checks passed |
+| Recent Progress | Started task157 from origin/main 0b31358436c38e698c7c2bc3a89871df273df21c; updating long-document SDG docs/comments to use NEMO_RUN_DIR-relative examples |

workspace/tasks/task157_data_sdg_long_document_docs_path_portability_s1/README.md

@@ -0,0 +1,18 @@
+# task157_data_sdg_long_document_docs_path_portability_s1
+
+## Scope
+
+- Replace concrete long-document SDG `/lustre/...` documentation examples with `${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/...` examples.
+- Keep command names, option names, stage names, and manual/auto-serve semantics unchanged.
+- Add focused static coverage for the scoped README and OCR config comment.
+
+## Boundaries
+
+- No production Python, deployment YAML, CLI behavior, generated artifacts, or unrelated docs.
+- No live long-document SDG data prep, `--serve` launch, endpoint calls, downloads, training, eval, W&B, cluster jobs, deploy, artifact transfer, main push, or self-merge.
+
+## Status
+
+- Base: `0b31358436c38e698c7c2bc3a89871df273df21c`
+- Branch: `intern_nem_dev_1/task157_data_sdg_long_document_docs_path_portability_s1`
+- PR: pending

workspace/tasks/task157_data_sdg_long_document_docs_path_portability_s1/history_log.md

@@ -0,0 +1,7 @@
+# History Log
+
+## 2026-05-29
+
+- Started task157 from `origin/main` at `0b31358436c38e698c7c2bc3a89871df273df21c`.
+- Updated scoped long-document SDG docs/comments toward `${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/...` portability.
+- Added focused static docs test coverage.

workspace/tasks/task157_data_sdg_long_document_docs_path_portability_s1/task_knowledge.md

@@ -0,0 +1,4 @@
+# Task Knowledge
+
+- Long-document SDG examples should use `${NEMO_RUN_DIR:-.}/output/data/sdg/long-document/...` so docs do not steer users into named-user or cluster-specific storage.
+- The task is documentation/comment/static-test only; manual vLLM and `--serve` examples must remain recognizable and keep the same CLI options.