?raw import of large file with many newlines causes OOM (153x memory amplification from sourcemap generation)

[KTibow]

Describe the bug

Importing a large text file via ?raw causes the Vite dev server to crash with a V8 OOM (FATAL ERROR: Ineffective mark-compacts near heap limit, exit code 134).

The ?raw handler in the vite:asset plugin has no file size check and reads the entire file unconditionally:

// src/node/plugins/asset.ts - load hook
if (rawRE.test(id)) {
  const file = checkPublicFile(id, config) || cleanUrl(id);
  this.addWatchFile(file);
  return `export default ${JSON.stringify(await fsp.readFile(file, "utf-8"))}`;
}

This creates multiple in-memory copies (raw file + JSON.stringify result + template literal). The critical amplification comes after this hook returns: Vite's send() function generates a fallback source map via:

new MagicString(code).generateMap({
  source: path.basename(urlWithoutTimestamp),
  hires: "boundary",      // character-level mappings
  includeContent: true    // embeds full code in sourcesContent
})

For a file with millions of lines, the hires: "boundary" source map is enormous. The result is cached persistently in moduleGraph, so GC cannot reclaim it.

The newline ratio is the critical factor because it affects both JSON.stringify expansion (1 byte \n → 2 bytes \\n) and the number of source map boundary entries.

Measured impact (Vite 8.0.3, Node 24.10.0, 2048MB heap limit)

Response size multiplier (input → dev server response):

• 0% newlines: 2.3x (e.g. 100MB file → 233MB response)
• 10% newlines: 3.8x (e.g. 100MB file → 385MB response)
• 50% newlines: 10.3x (e.g. 20MB file → 207MB response)

OOM thresholds with 2048MB heap (NODE_OPTIONS="--max-old-space-size=2048"):

File Size	0% newlines	10% newlines	50% newlines
20MB	ok	ok	ok
25MB	—	—	OOM
50MB	ok	ok	OOM
100MB	ok	ok	OOM
110MB	—	OOM	—
150MB	ok	OOM	OOM
200MB	ok	OOM	OOM
300MB	ok	—	—

Approximate failure thresholds (2048MB heap):

• 0% newlines: >300MB
• 10% newlines: ~100–110MB
• 50% newlines: ~20–25MB

With the default Node heap (~1.5GB on this 14GB system), the thresholds would be lower. A 231MB file with ~50% newlines (the real-world case that triggered this investigation) OOMs reliably.

Workaround: Use ?url + fetch instead:

const filterUrls = import.meta.glob('./data/*.txt', { query: '?url' });
const mod = await filterUrls[path]();
const response = await fetch(mod.default);
const text = await response.text();

This is related to #11745 (closed, fixed in Vite 4.5) which addressed the same root cause (magic-string sourcemap OOM) but the fix reduced the multiplier from ~100x to ~5-7x, which is still fatal for moderately large files with many lines.

Reproduction

No StackBlitz link (too large for online repro). Minimal reproduction via shell:

npm create vite@latest repro -- --template vanilla && cd repro && npm install
mkdir data
# Generate 20MB file with 10M lines (50% newlines)
python3 -c "import sys; [sys.stdout.write('x\n') for _ in range(10_000_000)]" > data/test.txt

Add to src/main.js:

const loaders = import.meta.glob('../data/*.txt', { query: '?raw' });

Steps to reproduce

1. npm install
2. NODE_OPTIONS="--max-old-space-size=2048" npx vite
3. curl http://localhost:5173/data/test.txt?import&raw
4. Observe: server crashes with exit code 134, FATAL ERROR: Ineffective mark-compacts near heap limit

System Info

System:
    OS: Linux 7.0 Ubuntu 25.10 25.10 (Questing Quokka)
    CPU: (12) arm64 unknown
    Memory: 8.40 GB / 14.44 GB
    Container: Yes
    Shell: 5.2.37 - /bin/bash
Binaries:
    Node: 24.10.0
    npm: 11.6.1
    pnpm: 10.25.0

Used Package Manager

npm

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines
• Read the docs
• Check that there isn't already an issue that reports the same bug
• Make sure this is a Vite issue and not a framework-specific issue
• Check that this is a concrete bug
• The provided reproduction is a minimal reproducible example

---

This issue was drafted by GLM 5.1 with the reporter's review and approval.