Skip to content

Commit 5865dea

Browse files
Correct the Criteo paragraph against the benchmark logs
The largest streaming vs in-RAM posterior gap was 0.12 on a feature coefficient, not ~0.1 on the intercept (the intercept gap was 0.01), and the comparison ran on a 1M-row slice; the OOM extrapolation from the fitted slope is ~240M rows, not 250M.
1 parent 79fdc5c commit 5865dea

2 files changed

Lines changed: 8 additions & 6 deletions

File tree

examples/variational_inference/streaming_dataset.ipynb

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -405,9 +405,10 @@
405405
"standard, publicly available out-of-core learning benchmark. Peak memory for the\n",
406406
"streaming `DataLoader` stayed flat at about 0.7 GB across a sweep from 1M to 150M\n",
407407
"rows, while the in-RAM `pm.Minibatch` baseline rose linearly to 15.7 GB at 150M\n",
408-
"rows, which extrapolates to out-of-memory around 250M rows on the same 26 GB\n",
409-
"machine. The streaming and in-RAM posteriors agreed coefficient for coefficient;\n",
410-
"the largest gap was about 0.1, on the intercept.\n",
408+
"rows, which extrapolates to out-of-memory around 240M rows on the same 26 GB\n",
409+
"machine. On a 1M-row slice, where the in-RAM fit is cheap, the streaming and\n",
410+
"in-RAM posterior means agreed coefficient for coefficient (correlation 0.998\n",
411+
"across the 14 coefficients); the largest gap on any single coefficient was 0.12.\n",
411412
"\n",
412413
"## When to use it\n",
413414
"\n",

examples/variational_inference/streaming_dataset.myst.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -222,9 +222,10 @@ features plus the click label) on the
222222
standard, publicly available out-of-core learning benchmark. Peak memory for the
223223
streaming `DataLoader` stayed flat at about 0.7 GB across a sweep from 1M to 150M
224224
rows, while the in-RAM `pm.Minibatch` baseline rose linearly to 15.7 GB at 150M
225-
rows, which extrapolates to out-of-memory around 250M rows on the same 26 GB
226-
machine. The streaming and in-RAM posteriors agreed coefficient for coefficient;
227-
the largest gap was about 0.1, on the intercept.
225+
rows, which extrapolates to out-of-memory around 240M rows on the same 26 GB
226+
machine. On a 1M-row slice, where the in-RAM fit is cheap, the streaming and
227+
in-RAM posterior means agreed coefficient for coefficient (correlation 0.998
228+
across the 14 coefficients); the largest gap on any single coefficient was 0.12.
228229

229230
## When to use it
230231

0 commit comments

Comments
 (0)