Fixed #216.

Author: uhop

AGENTS.md

@@ -117,6 +117,7 @@ stream-json/
   - Options: `packKeys`, `packStrings`, `packNumbers`, `streamKeys`, `streamStrings`, `streamNumbers`, `jsonStreaming`.
 - **Assembler** (`src/assembler.js`, implementation in `src/core/assembler.js`) interprets the token stream and reconstructs JavaScript objects. Plain class — no `EventEmitter` inheritance in 3.x.
   - Used internally by all streamers via `streamBase`.
+  - Reads only packed tokens (`keyValue`, `stringValue`, `numberValue`); streamed chunks are ignored.
   - `Assembler.connectTo(stream, {onDone: asm => …})` is substrate-aware: accepts either a Node `Readable` (attaches `'data'` listener) or a Web `ReadableStream` (pumps via `getReader()`). Detection via `typeof stream.getReader === 'function'`. `asm.onDone(fn)` can set/clear the callback after construction.
   - For hot paths, prefer a manual `for await (const tok of readable) asm.consume(tok)` loop over `connectTo` — no async-closure overhead, errors propagate directly. `FlexAssembler` has the same shape.
   - `asm.tapChain` is a function for use in `chain()`.

ARCHITECTURE.md

@@ -150,6 +150,7 @@ All filters are built on `filterBase` (`src/filters/filter-base.js`):
 - It maintains a path stack tracking the current JSON position.
 - `filter` option: a string, RegExp, or function `(stack, chunk) → boolean` that determines whether to accept or reject each subobject.
 - `makeStackDiffer` generates structural tokens (start/end object/array, key tokens) to reconstruct the surrounding JSON envelope when filtering.
+- Keys are tracked only from `keyValue` tokens, so key-based paths and parent recreation need packed keys from upstream (the parser's default). Replayed parent keys are always packed; their streamed form mirrors upstream unless `streamKeys` is set.
 
 | Filter    | specialAction           | defaultAction  | Effect                              |
 | --------- | ----------------------- | -------------- | ----------------------------------- |
@@ -163,7 +164,7 @@ All filters are built on `filterBase` (`src/filters/filter-base.js`):
 All streamers are built on `streamBase` (`src/streamers/stream-base.js`):
 
 - `streamBase({push, first, level})` returns a factory that accepts `options` and returns a function for use in `chain()`.
-- Uses `Assembler` internally to reconstruct objects.
+- Uses `Assembler` internally to reconstruct objects; reads only packed tokens (`keyValue`, `stringValue`, `numberValue`), the parser's default.
 - `level` controls when to emit: level 0 for `streamValues`, level 1 for `streamArray`/`streamObject`.
 - `objectFilter` option enables early rejection: if `objectFilter(asm)` returns `false`, the object is abandoned without completing assembly.
 - `first` callback validates the opening token (e.g., `streamArray` requires `startArray`).

llms-full.txt

@@ -196,7 +196,7 @@ parserStream.on('data', token => console.log(token.name));
 
 ## Assembler
 
-`Assembler` — a plain class (no `EventEmitter` inheritance) that interprets the token stream and reconstructs JavaScript objects. 3.0 dropped the `'done'` event in favor of an `onDone` callback option.
+`Assembler` — a plain class (no `EventEmitter` inheritance) that interprets the token stream and reconstructs JavaScript objects. It reads only packed tokens (`keyValue`, `stringValue`, `numberValue`); streamed chunks are ignored, so the parser must pack keys, strings, and numbers (its default). 3.0 dropped the `'done'` event in favor of an `onDone` callback option.
 
 Constructor options:
 - `reviver` (function) — like `JSON.parse` reviver. Called as `reviver(key, value)`.
@@ -399,6 +399,9 @@ All filters are built on `filterBase` and accept these common options:
   - **RegExp** — matches when `regExp.test(stack.join(separator))`.
   - **function** `(stack, chunk) => boolean` — custom matching logic.
 - `pathSeparator` (string, default: `'.'`) — separator for path matching.
+- `streamKeys` (boolean; seeded by `streamValues`) — replay parent keys as `startKey`/`stringChunk`/`endKey` too. Default: mirrors upstream (on once streamed keys were received). Replayed keys are always emitted as `keyValue`.
+- `packKeys` — deprecated no-op on filters (still configures the parser in a `withParser()` bag).
+- Input requirement: key-based paths and parent recreation need packed keys (`keyValue`) from upstream — the parser's default.
 - `once` (boolean) — if true, stop filtering after the first match.
 - `maxDepth` (number, default: `1024`) — maximum JSON nesting depth to evaluate; a token nested deeper throws a `RangeError`. A guard for untrusted input with unbounded nesting. Pass `Infinity` to disable.
 - `streamKeys` (boolean) — control key streaming in output.
@@ -407,7 +410,7 @@ Each filter ships in both substrates. The Node entry (`stream-json/filters/<name
 
 ### pick(options)
 
-Passes only matching subobjects, discards everything else.
+Passes only matching subobjects, discards everything else. Key-based paths need packed keys (`keyValue`) from upstream — the parser's default.
 
 ```js
 import {pick} from 'stream-json/filters/pick.js';
@@ -424,7 +427,7 @@ const pipeline = pick.withParser({filter: 'data'});
 
 ### replace(options)
 
-Replaces matching subobjects with a replacement value.
+Replaces matching subobjects with a replacement value. Needs packed keys (`keyValue`) from upstream — the parser's default; replayed parent keys are always packed.
 
 Extra option:
 - `replacement` — the replacement:
@@ -450,7 +453,7 @@ chain([parser(), replace({
 
 ### ignore(options)
 
-Removes matching subobjects completely. A variant of Replace with `replacement = none`.
+Removes matching subobjects completely. A variant of Replace with `replacement = none`. Needs packed keys (`keyValue`) from upstream — the parser's default; replayed parent keys are always packed.
 
 ```js
 import {ignore} from 'stream-json/filters/ignore.js';
@@ -462,7 +465,7 @@ chain([parser(), ignore({filter: /^\d+\.extra\b/}), stringer()]);
 
 ### filter(options)
 
-Keeps matching subobjects while preserving the surrounding JSON structure.
+Keeps matching subobjects while preserving the surrounding JSON structure. Needs packed keys (`keyValue`) from upstream — the parser's default; replayed parent keys are always packed.
 
 Extra option:
 - `acceptObjects` (boolean) — if true, accepts entire objects (not just tokens).
@@ -507,11 +510,11 @@ const differ = makeStackDiffer(/* previousStack */ []);
 //   return differ(stack, chunk, options);
 ```
 
-The differ honors `streamKeys`, `streamValues`, `packKeys`, and `pathSeparator` from the filter's options.
+The differ honors `streamKeys`, `streamValues`, and `pathSeparator` from the filter's options. It always replays `keyValue`; the streamed form follows `streamKeys`, which by default mirrors upstream.
 
 ## Streamers
 
-All streamers are built on `streamBase` and produce `{key, value}` objects. Each is generic in the assembled value type — `streamArray<T>()`, `streamValues<T>()`, `streamObject<T>()` (and their `.withParser<T>()`) carry `T` through to the item's `value` field; the default is `unknown`. The item shape is the exported `KeyedValue<K, T>` type (`stream-json/streamers/stream-base.js`): `K` is `string` for `streamObject` (the property name) and `number` for `streamArray` (the array index) and `streamValues` (a sequential counter); `StreamArrayItem<T>` / `StreamObjectItem<T>` / `StreamValuesItem<T>` are aliases of it.
+All streamers are built on `streamBase` and produce `{key, value}` objects. They read only packed tokens (`keyValue`, `stringValue`, `numberValue`) — the parser's default. Each is generic in the assembled value type — `streamArray<T>()`, `streamValues<T>()`, `streamObject<T>()` (and their `.withParser<T>()`) carry `T` through to the item's `value` field; the default is `unknown`. The item shape is the exported `KeyedValue<K, T>` type (`stream-json/streamers/stream-base.js`): `K` is `string` for `streamObject` (the property name) and `number` for `streamArray` (the array index) and `streamValues` (a sequential counter); `StreamArrayItem<T>` / `StreamObjectItem<T>` / `StreamValuesItem<T>` are aliases of it.
 
 Common option:
 - `objectFilter` (function) `(asm) => boolean|null` — called during assembly. Return `true` to accept, `false` to reject (abandon assembly), `null`/`undefined` for undecided.
@@ -626,7 +629,7 @@ Creates a `gen(parser(options), fn(options))` pipeline — a function for use in
 
 Browser-safe Web-only entry: `stream-json/web/utils/with-parser.js` (has only `asWebStream`).
 
-Most components export `.withParser(options)`, `.withParserAsStream(options)`, and `.withParserAsWebStream(options)` static methods as a convenience:
+Most components export `.withParser(options)`, `.withParserAsStream(options)`, and `.withParserAsWebStream(options)` static methods as a convenience. They prepend `packKeys: true` to the options bag, so the parser packs keys even under `packValues: false` (filters and streamers need them); the bare utility adds nothing:
 
 ```js
 // These are equivalent:
@@ -683,7 +686,7 @@ fs.createReadStream('data.json').pipe(v);
 
 ### FlexAssembler
 
-Like Assembler but with custom containers (Map, Set, custom classes) at specific paths. Standalone clone — same API surface (`connectTo`, `tapChain`, `onDone`). `FlexAssembler.connectTo` is substrate-aware: accepts either a Node `Readable` or a Web `ReadableStream`.
+Like Assembler but with custom containers (Map, Set, custom classes) at specific paths. Reads only packed tokens, like Assembler. Standalone clone — same API surface (`connectTo`, `tapChain`, `onDone`). `FlexAssembler.connectTo` is substrate-aware: accepts either a Node `Readable` or a Web `ReadableStream`.
 
 Options:
 - `objectRules` — array of rules for objects: `{filter, create, add, finalize?}`.

llms.txt

@@ -67,7 +67,7 @@ For the SAX-style event API on Web, use the `EventTarget`-based variants from `s
 
 ### Assembler
 
-`Assembler` — class that reconstructs JS objects from tokens. Receives a per-value callback via the `onDone` option.
+`Assembler` — class that reconstructs JS objects from tokens. Reads only packed tokens (`keyValue`, `stringValue`, `numberValue`); streamed chunks are ignored. Receives a per-value callback via the `onDone` option.
 
 ```js
 import Assembler from 'stream-json/assembler.js';
@@ -118,7 +118,7 @@ e.addEventListener('keyValue', ev => console.log(ev.detail));
 
 ## Filters
 
-All filters accept `{filter, pathSeparator, once, streamKeys, maxDepth}` options. `filter` can be a string, RegExp, or `(stack, chunk) => boolean`. `maxDepth` caps the JSON nesting depth a filter evaluates (default `1024`; a deeper token throws a `RangeError`, `Infinity` disables the limit).
+All filters accept `{filter, pathSeparator, once, maxDepth, streamKeys}` options. Key-based paths and parent recreation need packed keys from upstream (the parser's default). Replayed parent keys are always packed; their streamed form mirrors upstream unless `streamKeys` says otherwise. `packKeys` on a filter is a deprecated no-op. `filter` can be a string, RegExp, or `(stack, chunk) => boolean`. `maxDepth` caps the JSON nesting depth a filter evaluates (default `1024`; a deeper token throws a `RangeError`, `Infinity` disables the limit).
 
 - **`pick(options)`** — passes only matching subobjects, discards the rest.
 - **`replace(options)`** — replaces matching subobjects. Extra option: `replacement` (function, token, or array of tokens).
@@ -143,7 +143,7 @@ chain([
 
 ## Streamers
 
-Assemble complete JS objects from a token stream. All produce `{key, value}` objects, generic in the assembled value type (`streamArray<T>()`, `streamValues<T>()`, `streamObject<T>()`; `value` defaults to `unknown`). The item shape is the exported `KeyedValue<K, T>` type — `key` is a `string` for `streamObject`, a `number` for the others.
+Assemble complete JS objects from a token stream. They read only packed tokens (`keyValue`, `stringValue`, `numberValue`) — the parser's default. All produce `{key, value}` objects, generic in the assembled value type (`streamArray<T>()`, `streamValues<T>()`, `streamObject<T>()`; `value` defaults to `unknown`). The item shape is the exported `KeyedValue<K, T>` type — `key` is a `string` for `streamObject`, a `number` for the others.
 
 - **`streamValues(options)`** — streams successive JSON values. Use with `jsonStreaming` or after `pick`.
 - **`streamArray(options)`** — streams elements of a single top-level array.

src/core/assembler.d.ts

@@ -33,6 +33,10 @@ export interface AssemblerOptions<T = unknown> {
  * `onDone` option (or call `.onDone(fn)`) to receive a callback each time a
  * top-level value is fully assembled.
  *
+ * Reads only packed tokens (`keyValue`, `stringValue`, `numberValue`); streamed
+ * chunks are ignored, so the upstream parser must pack keys, strings, and numbers
+ * (its default).
+ *
  * Generic in `T` (default `unknown`) — the type of the fully assembled value.
  * Declare `new Assembler<MyShape>()` to type `current` and `tapChain()`. Read
  * `current` in the `onDone` callback (when it actually holds the completed `T`);

src/core/filters/filter-base.d.ts

@@ -34,11 +34,19 @@ declare namespace filterBase {
      * Pass `Infinity` to disable the limit.
      */
     maxDepth?: number;
-    /** Initial value for `streamKeys`. Controls streaming of replayed keys. */
+    /** Initial value for `streamKeys`. */
     streamValues?: boolean;
-    /** Emit streaming key tokens (`startKey`/`stringChunk`/`endKey`) when replaying delayed keys. */
+    /**
+     * Replay parent keys as `startKey`/`stringChunk`/`endKey` too. Default: mirrors
+     * upstream — on once streamed keys have been received. Replayed keys are always
+     * emitted as `keyValue`.
+     */
     streamKeys?: boolean;
-    /** Expect packed `keyValue` tokens from upstream. */
+    /**
+     * @deprecated No effect on filters: keys are tracked only when they arrive as
+     * `keyValue`, and replayed keys are always packed. In a `withParser()` options
+     * bag it still configures the parser. To be removed in the next major.
+     */
     packKeys?: boolean;
   }
 

src/core/filters/filter-base.js

@@ -56,7 +56,8 @@ const filterBase =
       maxDepth = options?.maxDepth ?? DEFAULT_MAX_DEPTH;
     /** @type {(stack: any[], chunk?: any) => boolean} */
     let filter = defaultFilter;
-    let streamKeys = true;
+    let streamKeys = false,
+      mirrorStreamKeys = true;
     if (options) {
       if (typeof options.filter == 'function') {
         filter = options.filter;
@@ -65,8 +66,14 @@ const filterBase =
       } else if (options.filter instanceof RegExp) {
         filter = regExpFilter(options.filter, separator);
       }
-      if ('streamValues' in options) streamKeys = options.streamValues;
-      if ('streamKeys' in options) streamKeys = options.streamKeys;
+      if ('streamValues' in options) {
+        streamKeys = options.streamValues;
+        mirrorStreamKeys = false;
+      }
+      if ('streamKeys' in options) {
+        streamKeys = options.streamKeys;
+        mirrorStreamKeys = false;
+      }
     }
     const sanitizedOptions = {...options, filter, streamKeys, separator};
     let state = 'check',
@@ -204,6 +211,7 @@ const filterBase =
         switch (action) {
           case 'process-key':
             if (chunk.name === 'startKey') {
+              if (mirrorStreamKeys) sanitizedOptions.streamKeys = true; // replay keys in the forms received
               state = 'process-key';
               continue recheck;
             }
@@ -282,9 +290,7 @@ const makeStackDiffer =
           if (options?.streamKeys) {
             returnTokens.push({name: 'startKey'}, {name: 'stringChunk', value: key}, {name: 'endKey'});
           }
-          if (options?.packKeys || !options?.streamKeys) {
-            returnTokens.push({name: 'keyValue', value: key});
-          }
+          returnTokens.push({name: 'keyValue', value: key});
         } else if (typeof key == 'number' && options?.skippedArrayValue) {
           for (let i = Math.max(0, previousStack[commonLength] + 1); i < key; ++i) {
             returnTokens.push(...options.skippedArrayValue);
@@ -318,9 +324,7 @@ const makeStackDiffer =
         if (options?.streamKeys) {
           returnTokens.push({name: 'startKey'}, {name: 'stringChunk', value: key}, {name: 'endKey'});
         }
-        if (options?.packKeys || !options?.streamKeys) {
-          returnTokens.push({name: 'keyValue', value: key});
-        }
+        returnTokens.push({name: 'keyValue', value: key});
       }
     }
 

src/core/filters/filter.d.ts

@@ -4,6 +4,8 @@ import filterBase from './filter-base.js';
 
 /**
  * Filters subobjects from a token stream while preserving the original JSON shape.
+ * Needs packed keys (`keyValue`) from upstream to track paths and recreate parents — the
+ * parser's default; replayed parent keys are always packed.
  *
  * This is the pure, stream-agnostic factory — no `.asStream` / `.asWebStream` adapters
  * attached. For the Node-flavored entry (with both adapters) import from

src/core/filters/ignore.d.ts

@@ -4,6 +4,8 @@ import filterBase from './filter-base.js';
 
 /**
  * Removes matching subobjects from a token stream entirely.
+ * Needs packed keys (`keyValue`) from upstream to track paths and recreate parents — the
+ * parser's default; replayed parent keys are always packed.
  *
  * This is the pure, stream-agnostic factory — no `.asStream` / `.asWebStream` adapters
  * attached. For the Node-flavored entry (with both adapters) import from

src/core/filters/pick.d.ts

@@ -4,6 +4,7 @@ import filterBase from './filter-base.js';
 
 /**
  * Picks matching subobjects from a token stream, ignoring the rest.
+ * Key-based paths need packed keys (`keyValue`) from upstream — the parser's default.
  *
  * This is the pure, stream-agnostic factory — no `.asStream` / `.asWebStream` adapters
  * attached. For the Node-flavored entry (with both adapters) import from