Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
98f5581
native: add deparse
benasher44 Aug 12, 2026
6d5bc8c
native: bound the comment list and raise the deparse recursion limit
benasher44 Aug 13, 2026
fca1629
native: encode parse trees with protobufjs instead of protobuf-es
benasher44 Aug 13, 2026
6a95729
native: skip inherited properties when walking the parse tree
benasher44 Aug 13, 2026
210cc3b
native: validate numeric enum values too
benasher44 Aug 13, 2026
a2381b4
native: fix the imports in the README comment example
benasher44 Aug 13, 2026
0547ab0
native: reject inherited reverse mappings in the string enum path
benasher44 Aug 13, 2026
e6b233a
native: document the synchronous/unbounded characteristics for callers
benasher44 Aug 13, 2026
08c79ba
native: make every README example runnable, and test that they are
benasher44 Aug 13, 2026
5eaf49d
native: simplify the deparse code and consolidate its test corpus
benasher44 Aug 13, 2026
408e642
native: name the recorded encodings for what they are
benasher44 Aug 13, 2026
250c3af
native: correct the pin source named in generate-proto's header
benasher44 Aug 13, 2026
3e3a125
native: encode the remapped tree directly, without fromObject
jefflub-ashby Aug 14, 2026
912ef3c
native: write protobuf bytes in one pass instead of via protobufjs's …
benasher44 Aug 14, 2026
351cf30
native: fix unsigned 64-bit clamping and nulled message fields
benasher44 Aug 14, 2026
6cdb6ea
native: cache the field plan on the Type, as a null-prototype object
benasher44 Aug 14, 2026
8f99f9f
native: halve deparse by unpacking protobuf into the memory context
benasher44 Aug 14, 2026
9440fdf
native: apply patches with git apply, which exists where patch does not
benasher44 Aug 14, 2026
135d487
native: skip protobuf-c's per-message field loop when it has nothing …
benasher44 Aug 14, 2026
744d315
native: make the protobuf-c field-loop memo thread-local
benasher44 Aug 14, 2026
0e4eeb0
native: stop the deparser emitting E'' string literals
github-actions[bot] Aug 18, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 86 additions & 1 deletion native/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ supportedArchitectures:
Drop-in replacement for `@libpg-query/parser`:

```js
const { parse, parseSync, fingerprint, normalize, scan } = require('@ashbyhq/libpg-query-native');
const { parse, parseSync, deparse, deparseSync, fingerprint, normalize, scan } = require('@ashbyhq/libpg-query-native');

// Sync (no init needed — native loads instantly)
const result = parseSync('SELECT id, name FROM users WHERE active = true');
Expand All @@ -111,6 +111,91 @@ const result2 = await parse('SELECT id, name FROM users WHERE active = true');
| `fingerprintSync(sql)` / `fingerprint(sql)` | ✓ | ✓ | 16-char hex fingerprint |
| `normalizeSync(sql)` / `normalize(sql)` | ✓ | ✓ | Normalized query string |
| `scanSync(sql)` / `scan(sql)` | ✓ | ✓ | `ScanResult` with tokens |
| `deparseSync(tree, opts?)` / `deparse(tree, opts?)` | ✓ | ✓ | SQL string |
| `extractCommentsSync(sql)` / `extractComments(sql)` | ✓ | ✓ | `DeparseComment[]` |

### Deparsing

`deparse()` is the inverse of `parse()`, using PostgreSQL's own deparser rather than a
reimplementation of it — so its output tracks the server's grammar, and constructs added
in PG 18 round-trip instead of being silently dropped.

```js
const { parseSync, deparseSync } = require('@ashbyhq/libpg-query-native');

const tree = parseSync('select a,b from t where x=1');
deparseSync(tree);
// SELECT a, b FROM t WHERE x = 1

// Edit the tree in between to rewrite a query:
tree.stmts[0].stmt.SelectStmt.fromClause[0].RangeVar.relname = 'other_table';
deparseSync(tree);
// SELECT a, b FROM other_table WHERE x = 1
```

Encoding is strict: a misspelled field or a bogus enum value throws rather than being
dropped and deparsed into quietly wrong SQL. Trees the deparser itself rejects throw a
`SqlError` carrying the failing C function and line.

#### Formatting

`prettyPrint` breaks the statement across lines. The remaining layout options are
pretty-print options upstream, so they only take effect alongside it.

```js
deparseSync(tree, { prettyPrint: true, indentSize: 2 });
// SELECT a, b, c
// FROM mytable
// WHERE
// x = 1
// AND y = 2
```

| Option | Default | Description |
|--------|---------|-------------|
| `prettyPrint` | `false` | Break the statement across lines |
| `indentSize` | `4` | Spaces per indent level |
| `maxLineLength` | `80` | Soft wrap width for lists of items |
| `trailingNewline` | `false` | Append a newline after the statement |
| `commasStartOfLine` | `false` | Put separating commas at the start of the line |
| `comments` | — | Comments to weave back in (see below) |

#### Comments

Parse trees don't carry comments, so a parse/deparse round trip drops them. Pull them
off the source first and hand them back:

```js
const { extractCommentsSync } = require('@ashbyhq/libpg-query-native');

const sql = '-- keep me\nSELECT a FROM t';
deparseSync(parseSync(sql), { comments: extractCommentsSync(sql) });
// -- keep me
// SELECT a FROM t
```

Each comment carries `matchLocation` (the offset it anchors to), `newlinesBefore`,
`newlinesAfter` and `text` — filter or rewrite the list before passing it back.

#### How the tree gets to the deparser

`pg_query_deparse_protobuf()` takes a protobuf-encoded tree, but `parse()` returns JSON.
`pg_query.proto` maps between the two with `json_name` annotations — 1,683 of them, which
is why `SelectStmt` and `targetList` in the JSON correspond to `select_stmt` and
`target_list` in the schema. [protobufjs ignores
`json_name`](https://github.com/protobufjs/protobuf.js/pull/1825), so this used to be a
dead end; [`@bufbuild/protobuf`](https://github.com/bufbuild/protobuf-es) honours it.

The generated schema lives in `src/gen/pg_query_pb.ts` and is committed, so `npm ci` and
the platform builds need no protobuf toolchain. Regenerate it when the libpg_query pin
moves:

```bash
npm run generate:proto
```

That refuses to run unless `protos/18/pg_query.proto` matches the pinned tag — a tree
encoded against a mismatched schema would deparse into wrong SQL rather than fail loudly.

## Using jemalloc for optimal memory

Expand Down
7 changes: 7 additions & 0 deletions native/buf.gen.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
version: v2
plugins:
# Resolved from PATH — scripts/generate-proto.mjs prepends node_modules/.bin.
- local: protoc-gen-es
out: src/gen
opt:
- target=ts
243 changes: 243 additions & 0 deletions native/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading