Skip to content

Memory guards have no Ink dimension: render-path regressions merge green #3365

Description

@acoliver

Summary

The memory harness added by #2852 and extended by #3114 has three modes,
text, media and reasoning. All three drive PendingResponseBuffer and the
streaming pipeline. None of them renders through Ink. So the post-GC plateau
verdicts cannot observe a render-path regression, and the render path is the one
thing that runs continuously in the default configuration.

This is not hypothetical. The gap already produced a wrong conclusion, and
measuring the render path reversed it.

What the gap hid

While evaluating the Ink migration in #3345 I measured the styled-character
cache in isolation and concluded fork 7.1.0 was roughly 69 times better than the
pinned 6.4.8, which pointed towards a version bump as a memory win.

Rendering the actual alternate-buffer shape reversed that. Identical
18,000-frame workload per build, post-GC JSC heap in MB:

Turn fork 6.4.8 (pin) fork 7.1.0 upstream 7.1.1
1 116.36 84.36 67.31
2 114.70 159.75 92.15
3 114.64 236.84 108.55
4 115.56 312.10 141.26
5 115.46 387.42 156.68
6 115.19 464.47 173.18

The pinned fork plateaus within 1% across turns 2 to 6. Fork 7.1.0 grows about
76 MB per 3,000 frames and upstream 7.1.1 about 21 MB, neither settling.
Upstream also grows external linearly from 36.75 MB to 90.57 MB.

End-of-run native metrics from the same runs:

Metric fork 6.4.8 fork 7.1.0 upstream 7.1.1
WebKit Malloc virtual 5.0 G 5.0 G 5.4 G
WebKit Malloc dirty 198.0 MB 527.9 MB 897.8 MB
Physical footprint 230.3 MB 562.2 MB 930.8 MB

No existing check would have failed on any of this, because no guard renders
a frame. A dependency swap that introduced an unbounded render-path leak would
merge green.

Ink rendering alone also drives WebKit Malloc to roughly 5 GB virtual with no
streaming, media or shell activity, reproducing the allocator high-water shape
#2852 attributed elsewhere, from a second source nothing currently watches.

Proposal

Add an ink mode beside the existing three, reusing the harness rather than
replacing it:

  • Target (scripts/issue-2852-memory-target.ts): mount the alternate-buffer
    shape, a terminal-sized overflow: hidden root with no <Static>, matching
    AlternateBufferLayout. Render N frames of distinct <Text> content per turn.
    Checkpoint with Bun.gc(true) and sample bun:jsc heapStats() and
    process.memoryUsage(), exactly as the existing modes do.
  • Runner (scripts/issue-2852-memory-runner.ts): no change beyond accepting
    the mode. It already samples vmmap and footprint against the pid.
  • Verdict (scripts/issue-2852-memory-benchmark.ts): reuse
    evaluateMultiMetricPlateau unchanged. JSC heap, external and dirty WebKit
    Malloc are already the right metrics; upstream 7.1.1 fails on two of them.

A working probe already exists and is published in the #3345 evidence pack
(project-plans/issue3345/p0-spike-scripts.md, "Ink render-churn probe"). It
produced every number above. Promoting it into the harness is mostly wiring plus
the plateau assertion.

Why it is worth doing independently of #3345

  1. Any future Ink dependency change gets a regression gate. Both candidate
    upgrade targets currently regress, and that was invisible until measured.
  2. The render path runs continuously in the default configuration, so a leak
    there affects every interactive session.
  3. It closes the stated intent of Long-running sessions grow memory without bound until the host is exhausted (84.8 GB across sessions, 64 MB free observed) #3114, which added plateau verdicts for
    exactly these metrics but only over the streaming pipeline.

Caveats to carry into the implementation

  • The probe workload is adversarial: every line of every frame is distinct,
    which is worst case for any text cache. Build-to-build comparison is sound
    because the workload is identical, but a CI threshold needs a workload with
    realistic repetition, or a tolerance chosen against measured variance.
  • Figures above are one machine, one platform, one Bun version. The existing
    harness is already macOS-oriented for vmmap/footprint; the JSC-heap
    plateau assertion is portable and is the part worth gating on.
  • This measures JS-side allocation. It says nothing about IOAccelerator, which
    is Attribute and bound native IOSurface/IOAccelerator memory growth for image-bearing sessions (#2852 follow-up) #2905.

Related

Metadata

Metadata

Assignees

Labels

Ink UIthe packages/cli Ink based UI.ObservabilityIssues related to quotas being able to observe model or provider responses or behavior.

Type

Projects

Status
In Progress

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions