Skip to content

Commit d74229b

Browse files
committed
Updated docs and typings.
1 parent 688ab13 commit d74229b

15 files changed

Lines changed: 123 additions & 51 deletions

README.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,31 @@ pipeline.on('data', ({value}) => {
4040
pipeline.on('end', () => console.log(byDepartment));
4141
```
4242
43+
This works because `pick` selected a single array — `streamArray` then streams its elements. When `pick` matches **several** subobjects, its output is a sequence of separate sub-trees — structurally the same token stream a [JSON Streaming](https://en.wikipedia.org/wiki/JSON_Streaming) source produces — and [streamValues()](https://github.com/uhop/stream-json/wiki/StreamValues) assembles each match into its own object:
44+
45+
```js
46+
import {parser} from 'stream-json';
47+
import {pick} from 'stream-json/filters/pick.js';
48+
import {streamValues} from 'stream-json/streamers/stream-values.js';
49+
import chain from 'stream-chain';
50+
import fs from 'node:fs';
51+
52+
// depts.json: {"departments": [
53+
// {"name": "dev", "head": {"name": "Alice", "id": 1}, "staff": 20},
54+
// {"name": "ops", "head": {"name": "Bob", "id": 2}, "staff": 10}
55+
// ]}
56+
const pipeline = chain([
57+
fs.createReadStream('depts.json'),
58+
parser(),
59+
pick({filter: /^departments\.\d+\.head\b/}), // every department's "head" subobject
60+
streamValues() // assemble each picked sub-tree
61+
]);
62+
63+
pipeline.on('data', ({value}) => console.log(value.name));
64+
// → Alice
65+
// → Bob
66+
```
67+
4368
Each stage is a building block; `stream-chain` wires them into one stream and handles the streaming and backpressure. To read straight from a file you can drop `createReadStream` and use the Node-only [parseFile()](https://github.com/uhop/stream-json/wiki/parseFile); to write a stream back to disk, use [stringerToFile()](https://github.com/uhop/stream-json/wiki/stringerToFile). See [Recipes](https://github.com/uhop/stream-json/wiki/Recipes) for more.
4469
4570
## Installation

src/core/filters/filter-base.d.ts

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import {Flushable, Many, none} from 'stream-chain/defs.js';
1+
import {Many, none} from 'stream-chain/defs.js';
22
import parser from '../parser.js';
33

44
/**
@@ -10,9 +10,7 @@ import parser from '../parser.js';
1010
* @param config - Internal configuration defining filter behavior (actions, transitions).
1111
* @returns A factory that takes user-facing options and returns a flushable filter function.
1212
*/
13-
declare function filterBase(
14-
config?: filterBase.FilterBaseConfig
15-
): (options?: filterBase.FilterBaseOptions) => Flushable<parser.Token, parser.Token | Many<parser.Token> | typeof none>;
13+
declare function filterBase(config?: filterBase.FilterBaseConfig): (options?: filterBase.FilterBaseOptions) => parser.TokenTransform;
1614

1715
declare namespace filterBase {
1816
/** User-facing options shared by all filters. */

src/core/filters/filter.d.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ import filterBase from './filter-base.js';
1212
*
1313
* @param options - Filter options including `acceptObjects`.
1414
*/
15-
declare function filter(options?: filter.FilterOptions): Flushable<parser.Token, parser.Token | Many<parser.Token> | typeof none>;
15+
declare function filter(options?: filter.FilterOptions): parser.TokenTransform;
1616

1717
declare namespace filter {
1818
/** Options for `filter`, extending filter base options. */

src/core/filters/ignore.d.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ import filterBase from './filter-base.js';
1212
*
1313
* @param options - Filter options (`filter`, `once`, `pathSeparator`).
1414
*/
15-
declare function ignore(options?: filterBase.FilterBaseOptions): Flushable<parser.Token, parser.Token | Many<parser.Token> | typeof none>;
15+
declare function ignore(options?: filterBase.FilterBaseOptions): parser.TokenTransform;
1616

1717
declare namespace ignore {
1818
/** Creates a `parser() + ignore()` pipeline as a flushable function. */

src/core/filters/pick.d.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ import filterBase from './filter-base.js';
1212
*
1313
* @param options - Filter options (`filter`, `once`, `pathSeparator`).
1414
*/
15-
declare function pick(options?: filterBase.FilterBaseOptions): Flushable<parser.Token, parser.Token | Many<parser.Token> | typeof none>;
15+
declare function pick(options?: filterBase.FilterBaseOptions): parser.TokenTransform;
1616

1717
declare namespace pick {
1818
/** Creates a `parser() + pick()` pipeline as a flushable function. */

src/core/filters/replace.d.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ import filterBase from './filter-base.js';
1212
*
1313
* @param options - Filter and replacement options.
1414
*/
15-
declare function replace(options?: replace.ReplaceOptions): Flushable<parser.Token, parser.Token | Many<parser.Token> | typeof none>;
15+
declare function replace(options?: replace.ReplaceOptions): parser.TokenTransform;
1616

1717
declare namespace replace {
1818
/** Options for `replace`, extending filter base options with a replacement value. */

src/core/parser.d.ts

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ import {Flushable, Many, none} from 'stream-chain/defs.js';
1414
* @param options - Parser configuration including packing, streaming, and JSON streaming options.
1515
* @returns A flushable function for use in a `chain()` pipeline.
1616
*/
17-
declare function parser(options?: parser.ParserOptions): Flushable<string, Many<parser.Token> | typeof none>;
17+
declare function parser(options?: parser.ParserOptions): parser.TokenSource;
1818

1919
declare namespace parser {
2020
/**
@@ -70,14 +70,27 @@ declare namespace parser {
7070
jsonStreaming?: boolean;
7171
}
7272

73+
/** Stage shape of the parser: `text` → `tokens`. */
74+
export type TokenSource = Flushable<string, Many<Token> | typeof none>;
75+
/** Stage shape of the filters: `tokens` → `tokens`. */
76+
export type TokenTransform = Flushable<Token, Token | Many<Token> | typeof none>;
77+
/** Stage shape of the streamers: `tokens` → items (see `KeyedValue` in `streamers/stream-base`). */
78+
export type TokenConsumer<Item = unknown> = Flushable<Token, Item | Many<Item> | typeof none>;
79+
/** Stage shape of the stringers: `tokens` → `text`. */
80+
export type TokenStringer = Flushable<Token, string | typeof none>;
81+
7382
/** Self-reference for backwards compat: `import {parser} from 'stream-json/core/parser.js'`. */
7483
export const parser: typeof import('./parser.js').default;
7584
}
7685

7786
type Token = parser.Token;
7887
type TokenName = parser.TokenName;
7988
type ParserOptions = parser.ParserOptions;
89+
type TokenSource = parser.TokenSource;
90+
type TokenTransform = parser.TokenTransform;
91+
type TokenConsumer<Item = unknown> = parser.TokenConsumer<Item>;
92+
type TokenStringer = parser.TokenStringer;
8093

8194
export default parser;
8295
export {parser};
83-
export type {Token, TokenName, ParserOptions};
96+
export type {Token, TokenName, ParserOptions, TokenSource, TokenTransform, TokenConsumer, TokenStringer};

src/core/streamers/stream-array.d.ts

Lines changed: 5 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import {Flushable, Many, none} from 'stream-chain/defs.js';
22
import parser from '../parser.js';
3-
import type {StreamBaseOptions} from './stream-base.js';
3+
import type {KeyedValue, StreamBaseOptions} from './stream-base.js';
44

55
/**
66
* Streams elements of a top-level JSON array as `{key, value}` objects.
@@ -12,24 +12,18 @@ import type {StreamBaseOptions} from './stream-base.js';
1212
*
1313
* @param options - Streamer options (assembler settings, `objectFilter`).
1414
*/
15-
declare function streamArray<T = unknown>(
16-
options?: StreamBaseOptions
17-
): Flushable<parser.Token, streamArray.StreamArrayItem<T> | typeof none | Many<streamArray.StreamArrayItem<T>>>;
15+
declare function streamArray<T = unknown>(options?: StreamBaseOptions): parser.TokenConsumer<streamArray.StreamArrayItem<T>>;
1816

1917
declare namespace streamArray {
2018
/**
21-
* An item emitted by `streamArray`: the array index and its assembled value.
19+
* An item emitted by `streamArray` — `KeyedValue<number, T>`: `key` is the
20+
* zero-based array index, `value` the assembled element.
2221
*
2322
* Generic in `T` (default `unknown`). Declare `StreamArrayItem<MyRow>` to type
2423
* the `value` field; the streamer factory and `.withParser` carry the parameter
2524
* through.
2625
*/
27-
export interface StreamArrayItem<T = unknown> {
28-
/** Zero-based array index. */
29-
key: number;
30-
/** The fully assembled JavaScript value, typed as `T` (default `unknown`). */
31-
value: T;
32-
}
26+
export type StreamArrayItem<T = unknown> = KeyedValue<number, T>;
3327
/** Creates a `parser() + streamArray()` pipeline as a flushable function. */
3428
export function withParser<T = unknown>(
3529
options?: StreamBaseOptions & parser.ParserOptions

src/core/streamers/stream-base.d.ts

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,11 +36,26 @@ declare namespace streamBase {
3636
/** Include objects for which `objectFilter` never made a decision. Default: `false`. */
3737
includeUndecided?: boolean;
3838
}
39+
40+
/**
41+
* The common shape of every streamer's output item: a `{key, value}` pair.
42+
*
43+
* `K` is the key type: `string` for `streamObject` (the property name),
44+
* `number` for `streamArray` (the array index) and `streamValues` (a
45+
* sequential counter). `T` types the assembled `value`.
46+
*/
47+
export interface KeyedValue<K extends string | number = string | number, T = unknown> {
48+
/** The item's key: a property name, an array index, or a sequential counter. */
49+
key: K;
50+
/** The fully assembled JavaScript value, typed as `T` (default `unknown`). */
51+
value: T;
52+
}
3953
}
4054

4155
type StreamBaseConfig = streamBase.StreamBaseConfig;
4256
type StreamBaseOptions = streamBase.StreamBaseOptions;
57+
type KeyedValue<K extends string | number = string | number, T = unknown> = streamBase.KeyedValue<K, T>;
4358

4459
export default streamBase;
4560
export {streamBase};
46-
export type {StreamBaseConfig, StreamBaseOptions};
61+
export type {StreamBaseConfig, StreamBaseOptions, KeyedValue};

src/core/streamers/stream-object.d.ts

Lines changed: 5 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import {Flushable, Many, none} from 'stream-chain/defs.js';
22
import parser from '../parser.js';
3-
import type {StreamBaseOptions} from './stream-base.js';
3+
import type {KeyedValue, StreamBaseOptions} from './stream-base.js';
44

55
/**
66
* Streams top-level properties of a JSON object as `{key, value}` objects.
@@ -12,24 +12,18 @@ import type {StreamBaseOptions} from './stream-base.js';
1212
*
1313
* @param options - Streamer options (assembler settings, `objectFilter`).
1414
*/
15-
declare function streamObject<T = unknown>(
16-
options?: StreamBaseOptions
17-
): Flushable<parser.Token, streamObject.StreamObjectItem<T> | typeof none | Many<streamObject.StreamObjectItem<T>>>;
15+
declare function streamObject<T = unknown>(options?: StreamBaseOptions): parser.TokenConsumer<streamObject.StreamObjectItem<T>>;
1816

1917
declare namespace streamObject {
2018
/**
21-
* An item emitted by `streamObject`: the property key and its assembled value.
19+
* An item emitted by `streamObject` — `KeyedValue<string, T>`: `key` is the
20+
* property name, `value` the assembled property value.
2221
*
2322
* Generic in `T` (default `unknown`). Declare `StreamObjectItem<MyValue>` to
2423
* type the `value` field; the streamer factory and `.withParser` carry the
2524
* parameter through.
2625
*/
27-
export interface StreamObjectItem<T = unknown> {
28-
/** Object property name. */
29-
key: string;
30-
/** The fully assembled JavaScript value, typed as `T` (default `unknown`). */
31-
value: T;
32-
}
26+
export type StreamObjectItem<T = unknown> = KeyedValue<string, T>;
3327
/** Creates a `parser() + streamObject()` pipeline as a flushable function. */
3428
export function withParser<T = unknown>(
3529
options?: StreamBaseOptions & parser.ParserOptions

0 commit comments

Comments
 (0)