Skip to content

Add Execution Models docs and reorganize parallelism/GIL - #1593

Merged
mthrok merged 1 commit into
mainfrom
docs
Jul 17, 2026
Merged

Add Execution Models docs and reorganize parallelism/GIL#1593
mthrok merged 1 commit into
mainfrom
docs

Conversation

@mthrok

@mthrok mthrok commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Restructure the pipeline-parallelism documentation into a fundamentals page, a patterns page, and a GIL page, and align the surrounding guides.

  • Pipeline Parallelism (getting_started/parallelism.rst) is refocused as the fundamentals: the executor mechanism, dedicated thread pools (e.g. GPU transfer), the cost of crossing a process boundary, the .to() region API, and run_pipeline_in_subprocess. The obsolete "Multi-processing (combined)" section is removed; its point — a per-stage ProcessPoolExecutor pays a data round trip at every stage boundary — is folded into a new "cost of crossing a process boundary" subsection, which also covers cheap large-payload transfer (prefer torch.Tensor / NumPy arrays, which move through shared memory; wrap a raw byte string in a 1-D uint8 tensor; SharedMemorySegmentPool for the last bit of IPC performance). A note explains that a region stage's concurrency applies within each worker, so effective concurrency across the pool is concurrency × max_workers.
  • Execution Models (getting_started/execution_models.rst, new) documents the three patterns built on those primitives — MT, MTP, and MP — each with an ASCII diagram, a minimal PipelineBuilder snippet (using a continuous source), and guidance on when to use it. It introduces the noisy-neighbour effect (the loader's CPU work delaying the GPU kernel launches issued from the training process) and recommends MTP, and an intermediate-process MP, for GPU training.
  • Working Around the GIL (getting_started/gil.rst) replaces faq.rst (which had become GIL-only): which operations release the GIL, a pandas-vs-polars example (with a figure), what to do when a function does not release it, and which functions hold it.

Supporting changes:

  • The PipelineBuilder.to, SharedMemorySegmentPool, and SharedMemoryRingBuffer docstrings gain the region-concurrency note and links to the benchmark_arena_transport example / shared-memory-arena case study.
  • pytorch.rst links the new pages and closes with a short "isolate the loader (MTP)" pointer for production, while keeping its focus on decomposing a Dataset into stage functions.
  • The obsolete with pipeline.auto_stop() wrapper is removed from doc examples (the section that documents auto_stop itself is kept).

GIL guidance is de-duplicated so gil.rst is the single canonical source and the other pages link to it. Toctrees and cross-references are updated; the existing pipeline-parallelism and pipeline-parallelism-custom-mt labels are preserved so inbound links keep working.

Restructure the pipeline-parallelism documentation into a fundamentals page, a patterns page, and a GIL page, and align the surrounding guides.

- `Pipeline Parallelism` (`getting_started/parallelism.rst`) is refocused as the fundamentals: the executor mechanism, dedicated thread pools (e.g. GPU transfer), the cost of crossing a process boundary, the `.to()` region API, and `run_pipeline_in_subprocess`. The obsolete "Multi-processing (combined)" section is removed; its point — a per-stage `ProcessPoolExecutor` pays a data round trip at every stage boundary — is folded into a new "cost of crossing a process boundary" subsection, which also covers cheap large-payload transfer (prefer `torch.Tensor` / NumPy arrays, which move through shared memory; wrap a raw byte string in a 1-D `uint8` tensor; `SharedMemorySegmentPool` for the last bit of IPC performance). A note explains that a region stage's `concurrency` applies within each worker, so effective concurrency across the pool is `concurrency × max_workers`.
- `Execution Models` (`getting_started/execution_models.rst`, new) documents the three patterns built on those primitives — MT, MTP, and MP — each with an ASCII diagram, a minimal `PipelineBuilder` snippet (using a continuous source), and guidance on when to use it. It introduces the noisy-neighbour effect (the loader's CPU work delaying the GPU kernel launches issued from the training process) and recommends MTP, and an intermediate-process MP, for GPU training.
- `Working Around the GIL` (`getting_started/gil.rst`) replaces `faq.rst` (which had become GIL-only): which operations release the GIL, a pandas-vs-polars example (with a figure), what to do when a function does not release it, and which functions hold it.

Supporting changes:
- The `PipelineBuilder.to`, `SharedMemorySegmentPool`, and `SharedMemoryRingBuffer` docstrings gain the region-concurrency note and links to the `benchmark_arena_transport` example / shared-memory-arena case study.
- `pytorch.rst` links the new pages and closes with a short "isolate the loader (MTP)" pointer for production, while keeping its focus on decomposing a Dataset into stage functions.
- The obsolete `with pipeline.auto_stop()` wrapper is removed from doc examples (the section that documents `auto_stop` itself is kept).

GIL guidance is de-duplicated so `gil.rst` is the single canonical source and the other pages link to it. Toctrees and cross-references are updated; the existing `pipeline-parallelism` and `pipeline-parallelism-custom-mt` labels are preserved so inbound links keep working.
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Jul 17, 2026
@meta-codesync

meta-codesync Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

This pull request has been imported. If you are a Meta employee, you can view this in D112516147. (Because this pull request was imported automatically, there will not be any future comments.)

@mthrok
mthrok merged commit e9f6214 into main Jul 17, 2026
111 of 112 checks passed
@mthrok
mthrok deleted the docs branch July 17, 2026 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant