Skip to content

Commit 6c25ba1

Browse files
committed
Restore Model Training closing flow
1 parent ad020fa commit 6c25ba1

1 file changed

Lines changed: 0 additions & 4 deletions

File tree

book/quarto/contents/vol1/training/training.qmd

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6819,10 +6819,6 @@ The iron law classifies a slow job as compute-bound (improve arithmetic or algor
68196819

68206820
Training is where the iron law becomes a daily instrument. Mixed precision changes effective throughput and memory traffic; checkpointing exchanges memory for recomputation; scaling adds communication. The engineering task is to identify the dominant term and apply the least costly intervention that changes it.
68216821

6822-
```{=latex}
6823-
\newpage
6824-
```
6825-
68266822
::: {.callout-chapter-connection title="From training to data selection"}
68276823

68286824
Training produces learned parameters by spending compute on every example; the chapter's optimized run still requires `{python} GPT2SummaryChapterRecap.o_total_mem_gb_str` of memory. Once the training loop is efficient, the next question is whether every example earns that cost. Repeated or low-information samples consume the same forward pass, backward pass, optimizer update, and communication as examples that add coverage or correct a weakness.

0 commit comments

Comments
 (0)