Give the command-line host validate, render, transform and parse - #49
Merged
Merged
Conversation
Step 5 of specs/features/cli-and-host-seams.md, and the step the host
exists for: with `validate` a documentation repository is a CI gate, and
with `--partials` `{% partial %}` works outside a bespoke Rust host for the
first time.
`validate` prints one line per error, `path:line:column: level[id]:
message`, or with `--format json` one object per input in the shape the
WebAssembly bindings return, in bytes rather than UTF-16 units. It exits 1
on an error at level `error` or `critical` and prints the rest, because the
library's `error_level` exists to ship a rule that is surfaced but not yet
enforced. `render` prints HTML; `transform` and `parse` print the renderable
tree and the syntax tree as JSON, one value per input per line, so the
output composes with `jq`. The encoders are written by hand, iteratively,
in upstream's field order.
The configuration is the shared vocabulary read through `saphyr`, the YAML
reader the conformance harness already uses -- no serde -- with a YAML node
as a `Declaration`, JSON as the YAML it is, and `--var NAME=VALUE` read by
the same reader so that `count=3` is the number and `--var` and `--config`
can never disagree. Partials are every file under `--partials`, keyed by
relative path at any depth, read into a source arena the config borrows.
Eighteen integration tests drive the built binary over YAML and JSON
configurations, a nested partials directory, typed and quoted variables, a
hook refused with its path and this host's reason, and the exit code of
every path.
Eight findings from the review of #49, every one reproduced against the binary, and three the review cut for room. Two change what the output claims to be. `validate --format json` said it wrote the bindings' shape and wrote `column` where the bindings write `character` and `byteOffset`. It writes the bindings' positions now -- `character` and `offset` in UTF-16 code units, `byteOffset` in bytes -- from a per-document index, so a consumer written against either host reads the other. The human format counts its column in characters, as an editor does. `parse` said it matched `JSON.stringify(Markdoc.parse(source))` and matched neither the field order nor the omission of `tag` and `location` nor ECMAScript's number spelling; it does all three now, the numbers through the library's own coercion. The partials walk no longer aborts on an image beside the partials, and no longer follows a directory symlink -- a link back up the tree was walked until the file system gave up. A YAML key that is a number or a boolean is its text, found by text, where it used to become a sentinel that nothing could find; a key that is a list is refused by name. A second YAML document after `---` is refused rather than dropped. `--var =3`, which is what `--var $NAME=3` becomes when `NAME` is unset, is refused rather than declaring a variable called nothing. The three commands share their prologue, the YAML declaration borrows the document rather than cloning subtrees on every read, and `--help`'s first line says what the binary does rather than what it was going to do. Nine tests pin the findings; one is Unix-only, because it needs a symlink.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 5 of
specs/features/cli-and-host-seams.md, and the step the host exists for: withvalidatea documentation repository is a CI gate, and with--partials{% partial %}works outside a bespoke Rust host for the first time. No library change; conformance holds at95 green, 10 annotated, 0 failing (of 105).Commands
validatepath:line:column: level[id]: messageper error, or--format jsonrendertransformparsevalidateexits 1 only at levelerrororcritical; awarningis printed and passes, because the library'serror_levelexists to ship a rule that is surfaced but not yet enforced.--file LABELnames stdin in diagnostics. A file that cannot be read is reported and the run goes on (exit 2).Configuration
--config PATH— YAML or JSON, in the shared vocabulary. Read withsaphyr, the reader the conformance harness already uses (pure Rust, no serde); a YAML node implementsDeclaration, so keys are refused before values are read, and a hook is refused with its path and this host's reason ("a configuration file cannot hold code; keep the hook in a Rust host").--partials DIR— every file under the directory, at any depth, keyed by relative path with/. Read into aSourcesarena before the config is built, because aConfigborrows its partials' sources — the lifetime shape the spec warned would otherwise cost a rewrite.--var NAME=VALUE—VALUEread as YAML by the same reader as the file, socount=3is the number,name=xthe string,'v="3"'the string3, and--varand--configcan never disagree. Overrides the file's. Errors report atconfig.variables.NAME.JSON
Hand-written, iterative encoders in
json.rs— the library carries no serde, and the shapes are upstream's: a tag is{"$$mdtype":"Tag","name","attributes","children"}intag.tsorder, so this andJSON.stringifyover upstream's object produce the same bytes; a syntax-tree node carries every field including annotations ({type, name, value}) and parser errors; a validate error is the bindings'{type, lines, location?, error}with byte columns. Non-finite numbers arenull, asJSON.stringifyhas them. Decision 11 in the spec records the output-shape choices.Tests
18 in
tests/commands.rsover the built binary:tag-undefined(levelcritical, as upstream has it) without a config; a correct document passing with the YAML and with the JSON config; a missing required attribute by line and column;--format jsonas one object per input in the bindings' shape;--filelabelling stdin; an unreadable input reported and skipped; a hook refused with its path and reason; a missing config; a malformed--var; the configured element rendered; partials inlined from a nested directory; typed vars (n=3satisfiesequals($n, 3);n="3"does not) overriding the file; inputs concatenated in order;transformandparseshapes; JSON escaping. Nothing unwraps.Gates
CLI clippy and 15 + 18 tests, library clippy over both lanes and tests, vocabulary crate, wasm clippy, docs, standalone — all pass locally.
Review fixes
Eight findings from
/code-review, all reproduced against the binary, plus three the review cut for room — all taken, nine tests added (27 total; one Unix-only for the symlink):validate --format jsonwrites the bindings' positions exactly —line,characterandoffsetin UTF-16 code units,byteOffsetin bytes — from a per-documentUtf16Index, instead ofcolumnin bytes under a parity claim. The human column counts characters.parsematchesJSON.stringify(Markdoc.parse(src))— upstream's field order (attributesfirst after the marker),tag/locationomitted rather thannull, numbers in ECMAScript's spelling via the library's ownattribute_value, and\\b/\\fescapes.2024:,true:); a list or mapping key is refused by name. Tagged keys are found under their plain spelling.---is refused rather than dropped.--var =3is refused with the existingNAME=VALUEmessage.host::load/host::configured;Yaml<'a>(&'a YamlOwned)borrows instead of cloning subtrees on everyget/items; the manifest description no longer says the commands are "to follow"; the README no longer claimsfmttakes--file.