Skip to content

Commit a439233

Browse files
authored
Add docs block parsing to M1 manifest
Adds native Zig docs block parsing and literal doc() manifest support for the M1 parser slice.
1 parent d56f47a commit a439233

22 files changed

Lines changed: 366 additions & 19 deletions

File tree

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# M1 Docs Block Slice
2+
3+
## Scope
4+
5+
This slice extends the native Zig parser and partial manifest writer with a dbt Core-compatible docs block subset:
6+
7+
- discover `.md` files under configured model paths
8+
- parse `{% docs name %}` / `{% enddocs %}` blocks
9+
- emit `docs` manifest entries keyed as `doc.<package>.<name>`
10+
- resolve whole-description literal `{{ doc("name") }}` and `{{ doc('name') }}`
11+
- attach resolved doc block IDs to model and column `doc_blocks`
12+
- fail loudly for malformed docs blocks, duplicate docs names, missing docs, and dynamic `doc` calls
13+
14+
The product runtime remains Zig. Python changes are limited to tests.
15+
16+
## Compatibility Evidence
17+
18+
The fixture was checked against dbt Core 1.10 with the DuckDB adapter. The written Core manifest showed:
19+
20+
- docs entries include `unique_id`, `resource_type`, `package_name`, `name`, `path`, `original_file_path`, and `block_contents`
21+
- docs `path` is relative to the model path, while `original_file_path` includes the model path
22+
- docs block contents are trimmed before manifest emission
23+
- literal `doc` descriptions are rendered into model and column descriptions
24+
- model and column `doc_blocks` list referenced doc unique IDs
25+
- docs entries do not appear as lineage keys in `child_map`
26+
27+
Fusion preview was also checked as a secondary signal, but Core remains the contract for this M1 slice.
28+
29+
## Validation
30+
31+
Required before merging this slice:
32+
33+
- `zig fmt --check src/project.zig src/root.zig build.zig`
34+
- `zig build`
35+
- `zig build test`
36+
- `pytest -q`
37+
- `git diff --check`
38+
- public-safety and runtime-boundary scans
39+
- public Jaffle Shop DuckDB parse smoke, if the fixture checkout is available

PLAN.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -514,7 +514,7 @@ Exit criteria:
514514

515515
- M0 is under PR review as the Zig `0.16.0` runtime scaffold.
516516
- M1 has started on stacked branches with native Zig artifact-first parser slices.
517-
- `dxt parse` now targets the supported Tier 0 subset: project name/model paths/seed paths, SQL model discovery, CSV seed discovery, literal `ref` to models or seeds, literal `source`, inline `config(materialized=..., tags=...)`, narrow YAML model properties for scalar descriptions, simple columns, tags, materialization, disabled SQL models, dependency maps, and deterministic partial `manifest.json`. Generic tests are parsed for later graph work but are not yet emitted as dbt-shaped `test.*` nodes.
517+
- `dxt parse` now targets the supported Tier 0 subset: project name/model paths/seed paths, SQL model discovery, CSV seed discovery, docs block discovery, literal `ref` to models or seeds, literal `source`, literal `doc` in descriptions, inline `config(materialized=..., tags=...)`, narrow YAML model properties for scalar descriptions, simple columns, tags, materialization, disabled SQL models, dependency maps, and deterministic partial `manifest.json`. Generic tests are parsed for later graph work but are not yet emitted as dbt-shaped `test.*` nodes.
518518
- `dxt ls` now lists resources from the same parser graph with stable text/JSON output and basic name, tag, path, resource type, and exact exclude filters.
519-
- Synthetic fixtures cover one model, model refs, seed refs, source refs, combined source/model YAML, inline config/tag selection, YAML model properties and columns, disabled models, disabled ref diagnostics, unmatched model-property warnings, duplicate model diagnostics, unsupported dynamic ref diagnostics, and unsupported macro-call diagnostics.
520-
- The current M1 manual gate parses the public Jaffle Shop DuckDB project into a partial manifest with SQL models and CSV seeds. The next M1 slices should add published schema validation, docs blocks, macros, generic test nodes, stronger selector parity, and deeper Jaffle artifact parity.
519+
- Synthetic fixtures cover one model, model refs, seed refs, source refs, combined source/model YAML, inline config/tag selection, YAML model properties and columns, docs blocks with literal `doc` descriptions, disabled models, disabled ref diagnostics, unmatched model-property warnings, duplicate model and docs diagnostics, unsupported dynamic ref/doc diagnostics, missing doc diagnostics, malformed docs block diagnostics, and unsupported macro-call diagnostics.
520+
- The current M1 manual gate parses the public Jaffle Shop DuckDB project into a partial manifest with SQL models and CSV seeds. The next M1 slices should add published schema validation, macros, dbt-shaped generic test nodes, stronger selector parity, and deeper Jaffle artifact parity.

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ Implemented pre-alpha commands:
3535
./zig-out/bin/dxt ls --project-dir tests/fixtures/model_ref --output json
3636
```
3737

38-
`parse` and `ls` currently support only the documented M1 parser subset: `dbt_project.yml` name/model paths/seed paths/target path, SQL model discovery, CSV seed discovery, literal `ref` to models or seeds, literal `source`, basic inline `config`, narrow YAML model properties for scalar descriptions, simple columns, tags, materialization and disabled SQL models, deterministic partial `manifest.json`, and simple name/tag/path/resource filters. dbt-shaped generic test nodes, `compile`, `build`, and `docs generate` remain planned placeholders.
38+
`parse` and `ls` currently support only the documented M1 parser subset: `dbt_project.yml` name/model paths/seed paths/target path, SQL model discovery, CSV seed discovery, docs block discovery, literal `ref` to models or seeds, literal `source`, literal `doc` in descriptions, basic inline `config`, narrow YAML model properties for scalar descriptions, simple columns, tags, materialization and disabled SQL models, deterministic partial `manifest.json`, and simple name/tag/path/resource filters. dbt-shaped generic test nodes, `compile`, `build`, and `docs generate` remain planned placeholders.
3939

4040
## Development
4141

0 commit comments

Comments
 (0)