Add project dispatch config parsing (#67)

Author: sabino

.agent/research/dbt-upstream-reference-map.md

@@ -82,6 +82,11 @@ Every compatibility slice should record:
 - Existing extracted modules are `types`, `util`, `config`, `fs`, `jinja`,
   `loader`, `resolve`, `parse`, `selector`, `manifest`, `compiler`, and
   `catalog`.
+- Root-project `dispatch:` config is parsed in `src/project/config.zig`, copied
+  into the graph by `src/project/loader.zig`, and consumed by
+  `src/project/resolve.zig` for static `adapter.dispatch(...)`
+  `depends_on.macros` extraction. This is documented in
+  `.agent/research/m2-project-dispatch-config.md`.
 - The test base includes native Zig tests for module-level helpers and pytest
   integration tests for CLI/artifact fixtures plus a pinned local Manifest v12
   schema slice.

.agent/research/m2-project-dispatch-config.md

@@ -0,0 +1,86 @@
+# M2 Project Dispatch Config Slice
+
+This slice adds root-project `dbt_project.yml` `dispatch:` parsing for static
+`adapter.dispatch(...)` dependency extraction. It records `depends_on.macros`
+only. It does not execute dispatched macros, implement macro runtime,
+materializations, bundled dbt internal macros, adapter connections, or dynamic
+dispatch names/namespaces.
+
+## Upstream References
+
+dbt Core v1, branch `1.latest`, commit `566b75d`:
+
+- `core/dbt/contracts/project.py::Project` defines `dispatch` as a list and
+  validates each non-empty entry requires `macro_namespace` and list-valued
+  `search_order`.
+- `core/dbt/config/project.py::Project.from_project_config` carries
+  `cfg.dispatch` into runtime project config.
+- `core/dbt/config/project.py::Project.get_macro_search_order` returns the
+  first matching entry's `search_order` for a namespace.
+- `core/dbt/context/providers.py::BaseDatabaseWrapper._get_search_packages`
+  applies dispatch config before dependency fallback. No namespace searches the
+  flattened namespace, configured namespace search order wins, dependency
+  namespaces fall back to root project then dependency package, and empty search
+  order behaves like no configured order because dbt Core checks truthiness.
+- `core/dbt/context/providers.py::BaseDatabaseWrapper.dispatch` rejects dotted
+  macro names and deprecated `packages`, then loops packages first and adapter
+  prefixes second for `{prefix}__{macro_name}`.
+
+dbt Core v2 / Fusion foundation, branch `main`, commit `0529e06`:
+
+- `crates/dbt-schemas/src/schemas/project/dbt_project.rs::_Dispatch` models
+  `macro_namespace: String` and `search_order: Vec<String>`.
+- `crates/dbt-loader/src/loader.rs` loads root-project dispatch config into the
+  global `DISPATCH_CONFIG` map.
+- `crates/dbt-jinja-utils/src/phases/compile_and_run_context.rs` exposes that
+  map as `MACRO_DISPATCH_ORDER` using typed `Vec<String>` values.
+- `crates/dbt-jinja/minijinja/src/dispatch_object.rs::DispatchObject::get_search_packages`
+  uses configured dispatch order before dependency fallback.
+- `dispatch_object.rs::get_adapter_prefixes` preserves the adapter prefix
+  fallback shape used by the prior profile-derived adapter identity slice.
+
+## dxt Ownership
+
+- `src/project/config.zig` parses the narrow root-project `dispatch:` YAML
+  surface.
+- `src/project/types.zig` stores `DispatchConfig` on `ProjectConfig` and
+  `Graph`.
+- `src/project/loader.zig` copies only root project dispatch config into the
+  graph before macro and SQL scanning. Installed package `dispatch:` does not
+  affect root dispatch config in this slice.
+- `src/project/resolve.zig` applies configured package search order before the
+  existing dependency fallback.
+- `src/project/jinja.zig` continues to recognize literal
+  `adapter.dispatch(...)` and records the resolved macro ID.
+
+## Supported Surface
+
+- Root `dbt_project.yml` block list:
+  - `macro_namespace: <string>`
+  - `search_order: [<package>, ...]`
+- Block-form `search_order` lists.
+- Empty inline `search_order: []` is accepted and follows dbt Core v1 behavior:
+  it falls through as no truthy configured order.
+
+Unsupported shapes fail through the existing narrow YAML boundary: missing
+`macro_namespace`, missing `search_order`, scalar `search_order`, extra dispatch
+entry keys, inline maps, dynamic dispatch values, deprecated `packages=`, and
+Jinja-rendered project config.
+
+## Validation
+
+- Native Zig tests cover dispatch config parsing, malformed config rejection,
+  configured search-order lookup, dbt Core v1 empty-list fallback, and SQL
+  scanner dependency recording.
+- Pytest fixture `adapter_dispatch_project_config` validates manifest
+  `depends_on.macros` through the Zig binary.
+- Manifest schema-slice validation remains the artifact gate.
+
+## Stop Conditions
+
+- Do not execute dispatched macros in this slice.
+- Do not implement general macro runtime or materialization lookup.
+- Do not add adapter connections, DuckDB execution, catalog introspection,
+  `run_results.json`, or `docs serve`.
+- Do not parse installed-package dispatch config as root config.
+- Do not add Python product runtime behavior.

PLAN.md

@@ -318,6 +318,17 @@ narrow scalar `profiles.yml` parser. This slice lets parse-time static
 profiles, validate credentials, read host-global profile locations, implement
 project `dispatch:` config, execute macros, or open adapter connections.
 
+Current project dispatch config source note:
+`.agent/research/m2-project-dispatch-config.md` maps upstream dbt Core v1
+project `dispatch:` validation, `get_macro_search_order`, and
+`BaseDatabaseWrapper.dispatch` package/prefix search order plus Fusion
+`DISPATCH_CONFIG` and `MACRO_DISPATCH_ORDER` behavior to dxt's narrow
+root-project dispatch config parser. This slice lets static
+`adapter.dispatch(...)` dependency extraction honor configured package
+`search_order` for a literal namespace. It does not execute dispatched macros,
+parse installed-package dispatch as root config, render Jinja in project config,
+or run adapters.
+
 The next source-grounded M1/M2 slices after macro block variant support are:
 
 1. Extend the render-only artifact boundary to adapter-free docs generation:
@@ -684,4 +695,4 @@ Exit criteria:
 - Selector wildcard behavior is currently pinned to observed dbt Core 1.10 behavior. dbt Fusion preview currently differs for resource-type-prefixed wildcard selectors such as `model.<package>.*` and filename-suffix path selectors such as `path:*orders.sql`; a future Fusion-compatibility slice must decide whether to support a selector dialect switch or a compatible superset.
 - Compatibility planning now uses a source-grounded reference map under `.agent/research/dbt-upstream-reference-map.md`; future feature slices should name upstream dbt v1/v2 source references, dxt Zig owners, affected artifact fields, validation gates, and stop conditions before implementation.
 - The committed dbt Core M1 oracle harness lives in `scripts/check_dbt_core_m1_oracle.py`. It is optional developer-side Python tooling that requires `dbt-core` and `dbt-duckdb`, invokes dbt Core through its Python runner, runs `dxt parse` through the Zig binary, and compares stable manifest slices for the supported synthetic M1 fixture ladder. It ignores dbt internal package docs/macros that are outside the current dxt artifact scope, records a known allowed gap for installed-package exposure refs that dbt Core resolves to a root same-name model while dxt currently resolves package-local, and leaves full source-map parity, full artifact schemas, and execution parity for later slices.
-- Before broadening M2 product implementation, close or explicitly re-scope the remaining M1 macro-compatibility behavior gaps. Macro `docs`/`meta` patch fields are covered for the current scalar artifact subset. Macro argument extraction under dbt Core v1 `flags.validate_macro_args` semantics and YAML patch argument validation/replacement are implemented for the manifest artifact surface. Static macro dependency lookup now uses the supported dbt order of current package, root project, other-package fallback for macro bodies, graph-present internal `dbt` macros, and literal `adapter.dispatch(...)` dependency extraction. Parse-time dispatch prefixes now come from a narrow source-grounded `profiles.yml` adapter identity parser and emit manifest `metadata.adapter_type`, with default DuckDB behavior preserved when no profile file is loaded. Macro execution, bundled dbt internal macros, project `dispatch:` config, full target context, credential validation, and materialization runtime lookup remain planned. `{% data_test %}` has native source-grounded parser coverage, but the local dbt Core 1.10 oracle rejects that tag before writing artifacts, so dbt-oracle coverage currently pins `{% test %}` and `{% materialization %}` block parity.
+- Before broadening M2 product implementation, close or explicitly re-scope the remaining M1 macro-compatibility behavior gaps. Macro `docs`/`meta` patch fields are covered for the current scalar artifact subset. Macro argument extraction under dbt Core v1 `flags.validate_macro_args` semantics and YAML patch argument validation/replacement are implemented for the manifest artifact surface. Static macro dependency lookup now uses the supported dbt order of current package, root project, other-package fallback for macro bodies, graph-present internal `dbt` macros, and literal `adapter.dispatch(...)` dependency extraction. Parse-time dispatch prefixes now come from a narrow source-grounded `profiles.yml` adapter identity parser and emit manifest `metadata.adapter_type`, with default DuckDB behavior preserved when no profile file is loaded. Root-project `dispatch:` config search order is now honored for static `adapter.dispatch(...)` dependency extraction. Macro execution, bundled dbt internal macros, full target context, credential validation, and materialization runtime lookup remain planned. `{% data_test %}` has native source-grounded parser coverage, but the local dbt Core 1.10 oracle rejects that tag before writing artifacts, so dbt-oracle coverage currently pins `{% test %}` and `{% materialization %}` block parity.

README.md

@@ -32,6 +32,7 @@ Implemented pre-alpha commands:
 ```sh
 ./zig-out/bin/dxt parse --project-dir tests/fixtures/model_ref --target-path target-dxt
 ./zig-out/bin/dxt parse --project-dir tests/fixtures/profile_adapter_dispatch --profiles-dir tests/fixtures/profile_adapter_dispatch --target-path target-dxt
+./zig-out/bin/dxt parse --project-dir tests/fixtures/adapter_dispatch_project_config --target-path target-dxt
 ./zig-out/bin/dxt ls --project-dir tests/fixtures/model_ref
 ./zig-out/bin/dxt ls --project-dir tests/fixtures/model_ref --output json
 ./zig-out/bin/dxt compile --project-dir tests/fixtures/compile_basic --target-path target-dxt
@@ -41,7 +42,7 @@ Implemented pre-alpha commands:
 ./zig-out/bin/dxt docs generate --project-dir tests/fixtures/docs_blocks --target-path target-dxt
 ```
 
-`parse`, `ls`, `compile`, `run`, `build`, and `docs generate` currently support only the documented M1/M2 parser and render-only subset: `dbt_project.yml` name/profile/model paths/seed paths/macro paths/target path, `flags.validate_macro_args` for macro manifest argument validation, narrow scalar `profiles.yml` adapter type selection for parse-time dispatch identity, and top-level scalar `vars`, CLI `--vars` scalar overrides, SQL model discovery, CSV seed discovery, installed package SQL model and CSV seed discovery from `dbt_packages`, source discovery, installed package source discovery, exposure discovery, installed package exposure discovery, project macro discovery, installed package macro discovery from `dbt_packages`, project and package docs block discovery, literal and narrow scalar `var('name')` / `var('name', 'default')`-backed `ref` to models or seeds, two-argument package refs, package-local refs in installed package models and exposures, dbt-style fallback from unqualified refs to a unique installed-package model or seed, literal and narrow scalar `var('name')` / `var('name', 'default')`-backed `source`, package-local sources in installed package models, dbt-style fallback from unqualified sources to a unique installed-package source, literal `doc` in descriptions, package-qualified, package-local, root-fallback, macro-body other-package fallback, graph-present internal `dbt` macro dependencies, and literal `adapter.dispatch(...)` macro dependencies using the selected adapter prefix plus `default` with source-grounded parent fallbacks for Redshift and Databricks, basic inline `config`, narrow project and package YAML model properties and macro properties including patched macro `docs`, scalar `meta`, YAML `arguments`, and dbt Core v1-style macro argument annotation warnings, project and package model/seed `+docs.node_color`, root-project model config overrides for installed packages, simple columns, tags, materialization and disabled SQL models, dbt-shaped generic test nodes for `unique`, `not_null`, `accepted_values`, and `relationships`, model/test `refs` and `sources` artifact fields, deterministic partial `manifest.json`, and basic name/tag/path/package/resource/config materialization selectors with exact `package:`/`package:this`, comma intersections, whitespace unions, multi-argument selector lists, repeated selector flags, and graph expansion. This is not full dbt `var()` or profile compatibility yet: `vars.yml`, nested/package-scoped vars, non-string values, `var.has_var`, Jinja-rendered var values, profile/project rendering with vars, credential validation, host-global profile lookup, and general `var()` usage remain planned. `compile` applies selectors/excludes to enabled SQL models, writes supported compiled SQL to `target/compiled/<package>/...`, and emits compile fields for compiled models. `run` and `build` are truthful execution preflight commands: they parse, resolve, apply selectors/excludes, compile supported selected models, write `manifest.json`, and then fail before execution with a clear adapter-runner boundary error. They do not run SQL, materialize relations, run tests, or write `run_results.json`. `docs generate` uses the same render-only compiler, writes `manifest.json`, writes compiled SQL for selected enabled SQL models, and emits an adapter-free empty `catalog.json` until relation introspection exists. The compiler currently renders only `config`, literal and narrow scalar var-backed `ref`, and literal and narrow scalar var-backed `source` calls; macro execution, project `dispatch:` config, full target context, materializations, tests, real `run`/`build` execution, non-empty catalog introspection, and `docs serve` remain planned.
+`parse`, `ls`, `compile`, `run`, `build`, and `docs generate` currently support only the documented M1/M2 parser and render-only subset: `dbt_project.yml` name/profile/model paths/seed paths/macro paths/target path, root-project `dispatch:` search order for static `adapter.dispatch(...)` dependency extraction, `flags.validate_macro_args` for macro manifest argument validation, narrow scalar `profiles.yml` adapter type selection for parse-time dispatch identity, and top-level scalar `vars`, CLI `--vars` scalar overrides, SQL model discovery, CSV seed discovery, installed package SQL model and CSV seed discovery from `dbt_packages`, source discovery, installed package source discovery, exposure discovery, installed package exposure discovery, project macro discovery, installed package macro discovery from `dbt_packages`, project and package docs block discovery, literal and narrow scalar `var('name')` / `var('name', 'default')`-backed `ref` to models or seeds, two-argument package refs, package-local refs in installed package models and exposures, dbt-style fallback from unqualified refs to a unique installed-package model or seed, literal and narrow scalar `var('name')` / `var('name', 'default')`-backed `source`, package-local sources in installed package models, dbt-style fallback from unqualified sources to a unique installed-package source, literal `doc` in descriptions, package-qualified, package-local, root-fallback, macro-body other-package fallback, graph-present internal `dbt` macro dependencies, and literal `adapter.dispatch(...)` macro dependencies using the selected adapter prefix plus `default` with source-grounded parent fallbacks for Redshift and Databricks, basic inline `config`, narrow project and package YAML model properties and macro properties including patched macro `docs`, scalar `meta`, YAML `arguments`, and dbt Core v1-style macro argument annotation warnings, project and package model/seed `+docs.node_color`, root-project model config overrides for installed packages, simple columns, tags, materialization and disabled SQL models, dbt-shaped generic test nodes for `unique`, `not_null`, `accepted_values`, and `relationships`, model/test `refs` and `sources` artifact fields, deterministic partial `manifest.json`, and basic name/tag/path/package/resource/config materialization selectors with exact `package:`/`package:this`, comma intersections, whitespace unions, multi-argument selector lists, repeated selector flags, and graph expansion. This is not full dbt `var()` or profile compatibility yet: `vars.yml`, nested/package-scoped vars, non-string values, `var.has_var`, Jinja-rendered var values, profile/project rendering with vars, credential validation, host-global profile lookup, and general `var()` usage remain planned. `compile` applies selectors/excludes to enabled SQL models, writes supported compiled SQL to `target/compiled/<package>/...`, and emits compile fields for compiled models. `run` and `build` are truthful execution preflight commands: they parse, resolve, apply selectors/excludes, compile supported selected models, write `manifest.json`, and then fail before execution with a clear adapter-runner boundary error. They do not run SQL, materialize relations, run tests, or write `run_results.json`. `docs generate` uses the same render-only compiler, writes `manifest.json`, writes compiled SQL for selected enabled SQL models, and emits an adapter-free empty `catalog.json` until relation introspection exists. The compiler currently renders only `config`, literal and narrow scalar var-backed `ref`, and literal and narrow scalar var-backed `source` calls; macro execution, dynamic dispatch, full target context, materializations, tests, real `run`/`build` execution, non-empty catalog introspection, and `docs serve` remain planned.
 
 ## Development