Add the type-erased arg-parser runtime kernel with property tests - #566
Merged
Conversation
Smaug123
force-pushed
the
argparser-erased-core
branch
3 times, most recently
from
July 14, 2026 07:59
268a9f8 to
332dc9b
Compare
Smaug123
force-pushed
the
argparser-erased-core
branch
3 times, most recently
from
July 14, 2026 20:57
805ef3e to
c844f1e
Compare
Owner
Author
|
Codex reviewed clean, by the way. Still got to use my human eyes. |
This was referenced Jul 14, 2026
Smaug123
force-pushed
the
argparser-erased-core
branch
from
July 15, 2026 08:07
02bec5b to
e9c5f43
Compare
ArgParserRuntime is the future core of the generated argument parser: a
pure, type-erased kernel (scan argv into an ordered event log; select
discriminated-union cases; validate against the schema shape) which will
be embedded verbatim into generated parser output and drive a thin typed
layer of converters and record assembly. It supports Leaf/Product/Sum
schemas: sums are the upcoming DU-alternative feature, selected
shape-first ("exactly one interpretation consumes every argument") with
globally-unique argument names.
Nothing consumes it yet; the generator cutover comes separately.
Tests include an executable reference semantics (expand every sum into
its complete alternatives; accept exactly one) and FsCheck properties:
- compositional selection == exhaustive alternative expansion, over
random schema trees and observed-sets fuzzed across valid/noisy/empty
regimes, with instrumented distribution assertions;
- scanning is lossless (unscan . scan = id) for arbitrary token lists;
- a trailing "--" appends exactly a Separator event;
- round trip: rendered command lines scan to exactly the intended
events, across =/space forms, case variation, negation, bare flags.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The selection semantics assume every addressable --token names at most one claimant under the scanner's case-insensitive matching; without that, matchLeaf silently routes colliding tokens to the first-declared leaf. Generation-time checks cannot see forms supplied via [<Literal>] constants, so generated code will re-check at runtime via WellFormedSchema.checkOrFail before parsing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ToUpperInvariant keying is strictly coarser than OrdinalIgnoreCase (the equality the scanner matches keys with): "s" and "ſ" uppercase to the same string but are distinct keys, so the checked constructor falsely rejected schemas the scanner routes unambiguously. Group claims with StringComparer.OrdinalIgnoreCase instead. Also reject forms no token can ever address: an empty form (its token is the positional separator) and forms containing '=' (a --key=value token splits at its first '='), either of which could otherwise leave a required argument, or a whole union case, permanently unsatisfiable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Smaug123
force-pushed
the
argparser-erased-core
branch
from
July 15, 2026 19:08
3748b8e to
f80f036
Compare
Smaug123
commented
Jul 15, 2026
Comment on lines
+160
to
+161
| /// True if this error means "stop the parse immediately" (historically these were | ||
| /// exceptions thrown mid-scan rather than accumulated). |
Owner
Author
There was a problem hiding this comment.
Suggested change
| /// True if this error means "stop the parse immediately" (historically these were | |
| /// exceptions thrown mid-scan rather than accumulated). | |
| /// True if this error means "stop the parse immediately". |
Smaug123
commented
Jul 15, 2026
|
|
||
| /// Scan argv into an ordered event log. Pure: performs no conversion, throws no exceptions. | ||
| /// | ||
| /// The grammar (deliberately preserving the historical parser's behaviour): |
Owner
Author
There was a problem hiding this comment.
Suggested change
| /// The grammar (deliberately preserving the historical parser's behaviour): | |
| /// The grammar: |
Smaug123
commented
Jul 15, 2026
| go ScanState.AwaitingKey (ScanEvent.Help arg :: acc) rest | ||
| else | ||
| // (The string overload: char literals do not survive the parse-and-reprint | ||
| // round trip through which this file is embedded into generated code.) |
Owner
Author
There was a problem hiding this comment.
I think this is a bug in Fantomas, by the way.
Smaug123
added a commit
that referenced
this pull request
Jul 15, 2026
Review edits which just missed the #566 squash: the isFatal and scan grammar docstrings no longer describe the pre-rewrite parser's behaviour as the reason for their shape. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #565. Part of the arg-parser rewrite working towards discriminated-union (sum) argument schemas.
What this is
ArgParserRuntimeis the future core of the generated argument parser: a pure, type-erased kernel that knows nothing about target types, only the shape of a schema (names, arity, repeatability, requiredness) and raw string tokens. It has three phases, all data-to-data:scan— tokenizes argv into an orderedScanEventlog (occurrences, positionals, errors, help, separator), preserving the historical grammar: greedy value consumption, boolean-like keys consuming only boolean literals,--no-negation, unknown-key laziness under[<PositionalArgs true>], and the Arg parser: a bare--no longer discards pending parser state #563 rule that--resolves pending state exactly like end-of-input.select— chooses a case for every Sum node from which leaves were observed: exactly one touched case wins; two is a conflict; zero falls back to the unique case satisfiable with no arguments.validate— missing-required / inactive-leaf / no-positional-sink structural errors, plus the list of leaves whose defaults must fire.Because selection is shape-first (never depends on value conversion), the typed layer that will follow — converters, defaults, record assembly, message rendering — cannot change which argument a token was routed to nor which union case is selected. That is what will make it safe to embed this file verbatim into generated output (no runtime dependency) with only thin generated code per type.
Nothing consumes this yet: the generator cutover is the next PR.
The interesting part: the tests
The exhaustive reference semantics is implemented in the test suite: expand every Sum into its complete alternatives (
P × (A + B) = P×A + P×B) and accept precisely those alternatives whose leaf set covers everything observed and whose required leaves were all observed; exactly one accepted alternative is a parse. FsCheck properties (with instrumented distribution assertions, so the tests fail if the generators stop exploring the interesting regimes):select≡ exhaustive expansion over random schema trees (fuzzed sum/product bias) and observed-sets biased from valid towards noisy/empty (fuzzed drop/optional/noise percentages), 2000 cases: unique-alternative ⟺ compositional accepts with identical choices and active leaves; zero alternatives ⟹ compositional errors; multiple ⟹ reported as ambiguity.unscan (scan args) = argsfor arbitrary token lists drawn from a mixed adversarial distribution (well/ill-formed keys, bool literals, separators,=-pathologies, unicode) — every input token is accounted for in exactly one event.--appends exactly aSeparatorevent (the Arg parser: a bare--no longer discards pending parser state #563 property, at the core level).=vs space, case flips, negation, bare flags with lookahead-aware fallback) scans to exactly the intended event list.Plus unit tests pinning the token grammar and the motivating DU example (
--foo=3vs--bar=8 --baz=9;--foo=3 --bar=8is a conflict;--bar=8selects BarCase and reports missing--baz).🤖 Generated with Claude Code