Conversation
Restructure the pipeline-parallelism documentation into a fundamentals page, a patterns page, and a GIL page, and align the surrounding guides. - `Pipeline Parallelism` (`getting_started/parallelism.rst`) is refocused as the fundamentals: the executor mechanism, dedicated thread pools (e.g. GPU transfer), the cost of crossing a process boundary, the `.to()` region API, and `run_pipeline_in_subprocess`. The obsolete "Multi-processing (combined)" section is removed; its point — a per-stage `ProcessPoolExecutor` pays a data round trip at every stage boundary — is folded into a new "cost of crossing a process boundary" subsection, which also covers cheap large-payload transfer (prefer `torch.Tensor` / NumPy arrays, which move through shared memory; wrap a raw byte string in a 1-D `uint8` tensor; `SharedMemorySegmentPool` for the last bit of IPC performance). A note explains that a region stage's `concurrency` applies within each worker, so effective concurrency across the pool is `concurrency × max_workers`. - `Execution Models` (`getting_started/execution_models.rst`, new) documents the three patterns built on those primitives — MT, MTP, and MP — each with an ASCII diagram, a minimal `PipelineBuilder` snippet (using a continuous source), and guidance on when to use it. It introduces the noisy-neighbour effect (the loader's CPU work delaying the GPU kernel launches issued from the training process) and recommends MTP, and an intermediate-process MP, for GPU training. - `Working Around the GIL` (`getting_started/gil.rst`) replaces `faq.rst` (which had become GIL-only): which operations release the GIL, a pandas-vs-polars example (with a figure), what to do when a function does not release it, and which functions hold it. Supporting changes: - The `PipelineBuilder.to`, `SharedMemorySegmentPool`, and `SharedMemoryRingBuffer` docstrings gain the region-concurrency note and links to the `benchmark_arena_transport` example / shared-memory-arena case study. - `pytorch.rst` links the new pages and closes with a short "isolate the loader (MTP)" pointer for production, while keeping its focus on decomposing a Dataset into stage functions. - The obsolete `with pipeline.auto_stop()` wrapper is removed from doc examples (the section that documents `auto_stop` itself is kept). GIL guidance is de-duplicated so `gil.rst` is the single canonical source and the other pages link to it. Toctrees and cross-references are updated; the existing `pipeline-parallelism` and `pipeline-parallelism-custom-mt` labels are preserved so inbound links keep working.
Contributor
|
This pull request has been imported. If you are a Meta employee, you can view this in D112516147. (Because this pull request was imported automatically, there will not be any future comments.) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Restructure the pipeline-parallelism documentation into a fundamentals page, a patterns page, and a GIL page, and align the surrounding guides.
Pipeline Parallelism(getting_started/parallelism.rst) is refocused as the fundamentals: the executor mechanism, dedicated thread pools (e.g. GPU transfer), the cost of crossing a process boundary, the.to()region API, andrun_pipeline_in_subprocess. The obsolete "Multi-processing (combined)" section is removed; its point — a per-stageProcessPoolExecutorpays a data round trip at every stage boundary — is folded into a new "cost of crossing a process boundary" subsection, which also covers cheap large-payload transfer (prefertorch.Tensor/ NumPy arrays, which move through shared memory; wrap a raw byte string in a 1-Duint8tensor;SharedMemorySegmentPoolfor the last bit of IPC performance). A note explains that a region stage'sconcurrencyapplies within each worker, so effective concurrency across the pool isconcurrency × max_workers.Execution Models(getting_started/execution_models.rst, new) documents the three patterns built on those primitives — MT, MTP, and MP — each with an ASCII diagram, a minimalPipelineBuildersnippet (using a continuous source), and guidance on when to use it. It introduces the noisy-neighbour effect (the loader's CPU work delaying the GPU kernel launches issued from the training process) and recommends MTP, and an intermediate-process MP, for GPU training.Working Around the GIL(getting_started/gil.rst) replacesfaq.rst(which had become GIL-only): which operations release the GIL, a pandas-vs-polars example (with a figure), what to do when a function does not release it, and which functions hold it.Supporting changes:
PipelineBuilder.to,SharedMemorySegmentPool, andSharedMemoryRingBufferdocstrings gain the region-concurrency note and links to thebenchmark_arena_transportexample / shared-memory-arena case study.pytorch.rstlinks the new pages and closes with a short "isolate the loader (MTP)" pointer for production, while keeping its focus on decomposing a Dataset into stage functions.with pipeline.auto_stop()wrapper is removed from doc examples (the section that documentsauto_stopitself is kept).GIL guidance is de-duplicated so
gil.rstis the single canonical source and the other pages link to it. Toctrees and cross-references are updated; the existingpipeline-parallelismandpipeline-parallelism-custom-mtlabels are preserved so inbound links keep working.