Skip to content

Give the command-line host validate, render, transform and parse - #49

Merged
zoosky merged 3 commits into
mainfrom
feature/cli-commands
Sep 12, 2026
Merged

zoosky merged 3 commits into
mainfrom
feature/cli-commands

Conversation

@zoosky

@zoosky zoosky commented Sep 12, 2026

Copy link
Copy Markdown
Owner

Step 5 of specs/features/cli-and-host-seams.md, and the step the host exists for: with validate a documentation repository is a CI gate, and with --partials {% partial %} works outside a bespoke Rust host for the first time. No library change; conformance holds at 95 green, 10 annotated, 0 failing (of 105).

Commands

Command Does Needs configuration
validate path:line:column: level[id]: message per error, or --format json for tags of your own
render HTML, inputs concatenated for tags of your own
transform the renderable tree as JSON, one array per input per line for tags of your own
parse the syntax tree as JSON, one value per input per line no

validate exits 1 only at level error or critical; a warning is printed and passes, because the library's error_level exists to ship a rule that is surfaced but not yet enforced. --file LABEL names stdin in diagnostics. A file that cannot be read is reported and the run goes on (exit 2).

Configuration

  • --config PATH — YAML or JSON, in the shared vocabulary. Read with saphyr, the reader the conformance harness already uses (pure Rust, no serde); a YAML node implements Declaration, so keys are refused before values are read, and a hook is refused with its path and this host's reason ("a configuration file cannot hold code; keep the hook in a Rust host").
  • --partials DIR — every file under the directory, at any depth, keyed by relative path with /. Read into a Sources arena before the config is built, because a Config borrows its partials' sources — the lifetime shape the spec warned would otherwise cost a rewrite.
  • --var NAME=VALUEVALUE read as YAML by the same reader as the file, so count=3 is the number, name=x the string, 'v="3"' the string 3, and --var and --config can never disagree. Overrides the file's. Errors report at config.variables.NAME.

JSON

Hand-written, iterative encoders in json.rs — the library carries no serde, and the shapes are upstream's: a tag is {"$$mdtype":"Tag","name","attributes","children"} in tag.ts order, so this and JSON.stringify over upstream's object produce the same bytes; a syntax-tree node carries every field including annotations ({type, name, value}) and parser errors; a validate error is the bindings' {type, lines, location?, error} with byte columns. Non-finite numbers are null, as JSON.stringify has them. Decision 11 in the spec records the output-shape choices.

Tests

18 in tests/commands.rs over the built binary: tag-undefined (level critical, as upstream has it) without a config; a correct document passing with the YAML and with the JSON config; a missing required attribute by line and column; --format json as one object per input in the bindings' shape; --file labelling stdin; an unreadable input reported and skipped; a hook refused with its path and reason; a missing config; a malformed --var; the configured element rendered; partials inlined from a nested directory; typed vars (n=3 satisfies equals($n, 3); n="3" does not) overriding the file; inputs concatenated in order; transform and parse shapes; JSON escaping. Nothing unwraps.

Gates

CLI clippy and 15 + 18 tests, library clippy over both lanes and tests, vocabulary crate, wasm clippy, docs, standalone — all pass locally.

Review fixes

Eight findings from /code-review, all reproduced against the binary, plus three the review cut for room — all taken, nine tests added (27 total; one Unix-only for the symlink):

  • validate --format json writes the bindings' positions exactlyline, character and offset in UTF-16 code units, byteOffset in bytes — from a per-document Utf16Index, instead of column in bytes under a parity claim. The human column counts characters.
  • parse matches JSON.stringify(Markdoc.parse(src)) — upstream's field order (attributes first after the marker), tag/location omitted rather than null, numbers in ECMAScript's spelling via the library's own attribute_value, and \\b/\\f escapes.
  • Partials: a non-UTF-8 file is passed over rather than aborting the run; a directory symlink is not followed (a file symlink is read); the README says both.
  • YAML keys: a scalar key is its text and is found by text (2024:, true:); a list or mapping key is refused by name. Tagged keys are found under their plain spelling.
  • One YAML document: a second after --- is refused rather than dropped.
  • --var =3 is refused with the existing NAME=VALUE message.
  • Cleanups: the three commands share host::load/host::configured; Yaml<'a>(&'a YamlOwned) borrows instead of cloning subtrees on every get/items; the manifest description no longer says the commands are "to follow"; the README no longer claims fmt takes --file.

Step 5 of specs/features/cli-and-host-seams.md, and the step the host
exists for: with `validate` a documentation repository is a CI gate, and
with `--partials` `{% partial %}` works outside a bespoke Rust host for the
first time.

`validate` prints one line per error, `path:line:column: level[id]:
message`, or with `--format json` one object per input in the shape the
WebAssembly bindings return, in bytes rather than UTF-16 units. It exits 1
on an error at level `error` or `critical` and prints the rest, because the
library's `error_level` exists to ship a rule that is surfaced but not yet
enforced. `render` prints HTML; `transform` and `parse` print the renderable
tree and the syntax tree as JSON, one value per input per line, so the
output composes with `jq`. The encoders are written by hand, iteratively,
in upstream's field order.

The configuration is the shared vocabulary read through `saphyr`, the YAML
reader the conformance harness already uses -- no serde -- with a YAML node
as a `Declaration`, JSON as the YAML it is, and `--var NAME=VALUE` read by
the same reader so that `count=3` is the number and `--var` and `--config`
can never disagree. Partials are every file under `--partials`, keyed by
relative path at any depth, read into a source arena the config borrows.

Eighteen integration tests drive the built binary over YAML and JSON
configurations, a nested partials directory, typed and quoted variables, a
hook refused with its path and this host's reason, and the exit code of
every path.
Eight findings from the review of #49, every one reproduced against the
binary, and three the review cut for room. Two change what the output
claims to be.

`validate --format json` said it wrote the bindings' shape and wrote
`column` where the bindings write `character` and `byteOffset`. It writes
the bindings' positions now -- `character` and `offset` in UTF-16 code
units, `byteOffset` in bytes -- from a per-document index, so a consumer
written against either host reads the other. The human format counts its
column in characters, as an editor does. `parse` said it matched
`JSON.stringify(Markdoc.parse(source))` and matched neither the field order
nor the omission of `tag` and `location` nor ECMAScript's number spelling;
it does all three now, the numbers through the library's own coercion.

The partials walk no longer aborts on an image beside the partials, and no
longer follows a directory symlink -- a link back up the tree was walked
until the file system gave up. A YAML key that is a number or a boolean is
its text, found by text, where it used to become a sentinel that nothing
could find; a key that is a list is refused by name. A second YAML document
after `---` is refused rather than dropped. `--var =3`, which is what
`--var $NAME=3` becomes when `NAME` is unset, is refused rather than
declaring a variable called nothing.

The three commands share their prologue, the YAML declaration borrows the
document rather than cloning subtrees on every read, and `--help`'s first
line says what the binary does rather than what it was going to do. Nine
tests pin the findings; one is Unix-only, because it needs a symlink.
Steps 1 to 5 landed in #45 through #49. What step 6 leaves is the release
itself: the three manifests to 0.11.0 and Unreleased into a dated section,
which release.sh checks.
@zoosky
zoosky merged commit e351781 into main Sep 12, 2026
13 checks passed
@zoosky
zoosky deleted the feature/cli-commands branch September 12, 2026 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant