Extract macro property application

Author: sabino

PLAN.md

@@ -569,7 +569,7 @@ Exit criteria:
 
 - M0 is complete as the Zig `0.16.0` runtime scaffold.
 - M1 has started on stacked branches with native Zig artifact-first parser slices.
-- M1A has started with behavior-preserving `src/project/types.zig`, `src/project/selector.zig`, `src/project/manifest.zig`, `src/project/util.zig`, `src/project/config.zig`, `src/project/fs.zig`, `src/project/jinja.zig`, `src/project/resolve.zig`, and `src/project/parse.zig` extractions. `src/project/manifest.zig` owns selected-resource JSON and partial `manifest.json` writing with native tests for selected JSON shape, JSON escaping, exposure dependency ordering, disabled-resource filtering, and graph-map output. `src/project/util.zig` owns shared display, membership, append-dedup, string sorting, and narrow YAML scalar/list helpers. `src/project/config.zig` owns `dbt_project.yml` loading, project path/docs config parsing, and applying parsed project path/docs configs to graph nodes. `src/project/fs.zig` owns deterministic resource file discovery, Linux directory traversal helpers, and resource path/name helpers. `src/project/jinja.zig` owns lexical Jinja call, parenthesis, quoted string, literal argument helpers, supported model SQL scanning, inline config/tag parsing, and known macro-call scanning with native tests. `src/project/resolve.zig` owns graph lookup/count helpers, canonical graph resource ordering, duplicate resource validation, macro unique-id package extraction, low-level ref/source resolution helpers, and dependency-map mutation for refs, sources, and known macro dependencies. `src/project/parse.zig` owns narrow parser scalar helpers for YAML booleans, JSON-compatible scalar classification, generic-test YAML item names, generic-test definition construction/cloning, generic-test relationship target ref parsing, and generic-test identity/name/hash helpers with native tests. `src/project.zig` remains the public parser/list facade and still owns most loader, resource parser, and resolver orchestration code until follow-up extractions move those pieces behind focused internal modules.
+- M1A has started with behavior-preserving `src/project/types.zig`, `src/project/selector.zig`, `src/project/manifest.zig`, `src/project/util.zig`, `src/project/config.zig`, `src/project/fs.zig`, `src/project/jinja.zig`, `src/project/resolve.zig`, and `src/project/parse.zig` extractions. `src/project/manifest.zig` owns selected-resource JSON and partial `manifest.json` writing with native tests for selected JSON shape, JSON escaping, exposure dependency ordering, disabled-resource filtering, and graph-map output. `src/project/util.zig` owns shared display, membership, append-dedup, string sorting, and narrow YAML scalar/list helpers. `src/project/config.zig` owns `dbt_project.yml` loading, project path/docs config parsing, and applying parsed project path/docs configs to graph nodes. `src/project/fs.zig` owns deterministic resource file discovery, Linux directory traversal helpers, and resource path/name helpers. `src/project/jinja.zig` owns lexical Jinja call, parenthesis, quoted string, literal argument helpers, supported model SQL scanning, inline config/tag parsing, and known macro-call scanning with native tests. `src/project/resolve.zig` owns graph lookup/count helpers, canonical graph resource ordering, duplicate resource validation, macro unique-id package extraction, low-level ref/source resolution helpers, and dependency-map mutation for refs, sources, and known macro dependencies. `src/project/parse.zig` owns narrow parser scalar helpers for YAML booleans, JSON-compatible scalar classification, generic-test YAML item names, generic-test definition construction/cloning, generic-test relationship target ref parsing, macro-property application, and generic-test identity/name/hash helpers with native tests. `src/project.zig` remains the public parser/list facade and still owns most loader, resource parser, and resolver orchestration code until follow-up extractions move those pieces behind focused internal modules.
 - `dxt parse` now targets the supported Tier 0 subset: project name/model paths/seed paths/macro paths, project and package model path configs for literal `+materialized`, `+tags`, and model/seed `+docs.node_color`, root-project model config overrides for installed packages, SQL model discovery, CSV seed discovery, installed package SQL model and CSV seed discovery from `dbt_packages`, source discovery, installed package source discovery, exposure discovery, installed package exposure discovery, project macro discovery, installed package macro discovery from `dbt_packages`, macro property YAML for project macro descriptions and arguments, project and package docs block discovery, literal `ref` to models or seeds, two-argument package refs, package-local refs in installed package models and exposures, unique installed-package fallback for unqualified refs, literal `source`, package-local sources in installed package models, unique installed-package fallback for unqualified sources, literal `doc` in project and package descriptions, inline `config(materialized=..., tags=...)`, known project/package-qualified/package-local macro call dependencies, narrow project and package YAML model properties for scalar descriptions, simple columns, tags, materialization, disabled SQL models, dbt-shaped `unique`, `not_null`, `accepted_values`, and `relationships` generic test nodes, model/test `refs` and `sources` artifact fields, dependency maps, and deterministic partial `manifest.json`. The manifest includes the v12 top-level maps needed by the M1 artifact shape and is covered by a pinned local dbt Manifest v12 schema slice. YAML generic test arguments are currently supported for scalar values plus inline and block lists required by public Jaffle Shop DuckDB-style tests.
 - `dxt ls` now lists dbt-selectable resources from the same parser graph with stable text/JSON output and basic name/FQN wildcards, tag wildcards, slash-aware `path:` wildcards, exact `package:`/`package:this`, `source:` wildcards including package-qualified source selectors, `exposure:` wildcards, `resource_type:`, `test_type:generic`, config materialization, comma intersection, whitespace union, multi-argument selector lists, repeated selector flags, leading/trailing `+` graph expansion, and exact exclude filters; macros are emitted in artifacts but not exposed as `ls` resources.
 - Synthetic fixtures cover one model, model refs, seed refs, source refs, exposure refs to models and sources, combined source/model YAML, inline config/tag selection, config materialization selection, comma-intersection selection, YAML model properties and columns, emitted `unique`, `not_null`, `accepted_values`, and `relationships` generic test nodes, project macro artifacts and macro properties, configured `macro-paths` replacing the default macro directory, installed package macros with package-qualified calls and package-local macro calls, installed package models, seeds, sources, docs, exposures, package YAML model properties, root package config overrides, and package-qualified/package-local refs/sources, macro calls recorded in model and macro `depends_on.macros`, docs blocks with literal `doc` descriptions, disabled models, disabled ref diagnostics, unmatched model-property warnings, duplicate model and docs diagnostics, unsupported dynamic ref/doc diagnostics, missing doc diagnostics, malformed docs block diagnostics, unresolved package macro diagnostics, and unsupported unknown macro-call diagnostics.

src/project.zig

@@ -23,9 +23,7 @@ const ColumnDef = types.ColumnDef;
 const GenericTestDef = types.GenericTestDef;
 const DocBlock = types.DocBlock;
 const MacroDef = types.MacroDef;
-const MacroArgument = types.MacroArgument;
 const ModelProperty = types.ModelProperty;
-const MacroProperty = types.MacroProperty;
 const Node = types.Node;
 const GenericTestNode = types.GenericTestNode;
 const Graph = types.Graph;
@@ -49,6 +47,7 @@ const splitKeyValue = util.splitKeyValue;
 const parseInlineStringList = util.parseInlineStringList;
 const dupTrimmedScalar = util.dupTrimmedScalar;
 const sortStrings = util.sortStrings;
+const applyMacroProperties = project_parse.applyMacroProperties;
 const appendGenericTestDef = project_parse.appendGenericTestDef;
 const appendGenericTestDefClone = project_parse.appendGenericTestDefClone;
 const parseBool = project_parse.parseBool;
@@ -68,7 +67,6 @@ const countActiveExposures = project_resolve.countActiveExposures;
 const countActiveNodes = project_resolve.countActiveNodes;
 const countActiveSeeds = project_resolve.countActiveSeeds;
 const findDoc = project_resolve.findDoc;
-const findMacroIndexByPackageAndName = project_resolve.findMacroIndexByPackageAndName;
 const findModelIndexByName = project_resolve.findModelIndexByName;
 const rejectDuplicateDocs = project_resolve.rejectDuplicateDocs;
 const rejectDuplicateExposures = project_resolve.rejectDuplicateExposures;
@@ -998,21 +996,6 @@ fn parseSeed(runtime: Runtime, seed_root: []const u8, relative_path: []const u8,
     try graph.nodes.append(runtime.allocator, node);
 }
 
-fn applyMacroProperties(graph: *Graph) !void {
-    for (graph.macro_properties.items) |property| {
-        const macro_index = findMacroIndexByPackageAndName(graph, property.package_name, property.name) orelse {
-            try graph.unmatched_macro_properties.append(graph.allocator, .{ .name = property.name, .patch_path = property.patch_path });
-            continue;
-        };
-        var macro = &graph.macros.items[macro_index];
-        macro.patch_path = property.patch_path;
-        if (property.description.len != 0) macro.description = property.description;
-        for (property.arguments.items) |argument| {
-            try appendMacroArgumentClone(graph, &macro.arguments, argument);
-        }
-    }
-}
-
 fn applyModelProperties(graph: *Graph, package_name: []const u8) !void {
     for (graph.model_properties.items) |property| {
         if (!std.mem.eql(u8, property.package_name, package_name)) continue;
@@ -1173,17 +1156,6 @@ fn resolveDocDescription(graph: *Graph, package_name: []const u8, description: [
     return doc.block_contents;
 }
 
-fn appendMacroArgumentClone(graph: *Graph, arguments: *std.ArrayList(MacroArgument), source: MacroArgument) !void {
-    for (arguments.items) |*existing| {
-        if (std.mem.eql(u8, existing.name, source.name)) {
-            if (source.type.len != 0) existing.type = source.type;
-            if (source.description.len != 0) existing.description = source.description;
-            return;
-        }
-    }
-    try arguments.append(graph.allocator, source);
-}
-
 fn appendMetaEntry(allocator: std.mem.Allocator, entries: *std.ArrayList(MetaEntry), key: []const u8, value: JsonScalar) !void {
     for (entries.items) |*existing| {
         if (std.mem.eql(u8, existing.key, key)) {

src/project/parse.zig

@@ -1,15 +1,18 @@
 const std = @import("std");
 const jinja = @import("jinja.zig");
+const resolve = @import("resolve.zig");
 const types = @import("types.zig");
 const util = @import("util.zig");
 
 const JsonScalar = types.JsonScalar;
 const GenericTestDef = types.GenericTestDef;
 const Graph = types.Graph;
+const MacroArgument = types.MacroArgument;
 const RefDep = types.RefDep;
 const dupTrimmedScalar = util.dupTrimmedScalar;
 const findMatchingParen = jinja.findMatchingParen;
 const parseLiteralArgs = jinja.parseLiteralArgs;
+const findMacroIndexByPackageAndName = resolve.findMacroIndexByPackageAndName;
 
 pub fn parseBool(value: []const u8) !bool {
     const trimmed = std.mem.trim(u8, value, " \t\r");
@@ -101,6 +104,32 @@ pub fn appendGenericTestDefClone(graph: *Graph, tests: *std.ArrayList(GenericTes
     try tests.append(graph.allocator, cloned);
 }
 
+pub fn applyMacroProperties(graph: *Graph) !void {
+    for (graph.macro_properties.items) |property| {
+        const macro_index = findMacroIndexByPackageAndName(graph, property.package_name, property.name) orelse {
+            try graph.unmatched_macro_properties.append(graph.allocator, .{ .name = property.name, .patch_path = property.patch_path });
+            continue;
+        };
+        var macro = &graph.macros.items[macro_index];
+        macro.patch_path = property.patch_path;
+        if (property.description.len != 0) macro.description = property.description;
+        for (property.arguments.items) |argument| {
+            try appendMacroArgumentClone(graph, &macro.arguments, argument);
+        }
+    }
+}
+
+fn appendMacroArgumentClone(graph: *Graph, arguments: *std.ArrayList(MacroArgument), source: MacroArgument) !void {
+    for (arguments.items) |*existing| {
+        if (std.mem.eql(u8, existing.name, source.name)) {
+            if (source.type.len != 0) existing.type = source.type;
+            if (source.description.len != 0) existing.description = source.description;
+            return;
+        }
+    }
+    try arguments.append(graph.allocator, source);
+}
+
 pub fn refDepFromValue(allocator: std.mem.Allocator, value: []const u8) !RefDep {
     const trimmed = std.mem.trim(u8, value, " \t\r");
     if (std.mem.startsWith(u8, trimmed, "ref(")) {
@@ -368,6 +397,78 @@ test "appendGenericTestDefClone copies nested accepted values list" {
     try std.testing.expectEqualStrings("returned", clones.items[0].accepted_values.items[1]);
 }
 
+test "applyMacroProperties applies descriptions patch paths and merges arguments" {
+    var arena = std.heap.ArenaAllocator.init(std.testing.allocator);
+    defer arena.deinit();
+    const allocator = arena.allocator();
+
+    var graph = Graph{ .allocator = allocator, .project_name = "demo" };
+    defer graph.deinit();
+
+    try graph.macros.append(allocator, .{
+        .unique_id = "macro.demo.format_id",
+        .package_name = "demo",
+        .name = "format_id",
+        .path = "macros/format_id.sql",
+        .original_file_path = "macros/format_id.sql",
+        .macro_sql = "",
+    });
+    try graph.macros.items[0].arguments.append(allocator, .{
+        .name = "column_name",
+        .type = "string",
+        .description = "",
+    });
+
+    try graph.macro_properties.append(allocator, .{
+        .package_name = "demo",
+        .name = "format_id",
+        .patch_path = "macros/schema.yml",
+        .description = "Formats an identifier.",
+    });
+    try graph.macro_properties.items[0].arguments.append(allocator, .{
+        .name = "column_name",
+        .type = "",
+        .description = "Identifier expression.",
+    });
+    try graph.macro_properties.items[0].arguments.append(allocator, .{
+        .name = "quote",
+        .type = "bool",
+        .description = "Whether to quote.",
+    });
+
+    try applyMacroProperties(&graph);
+
+    try std.testing.expectEqualStrings("macros/schema.yml", graph.macros.items[0].patch_path.?);
+    try std.testing.expectEqualStrings("Formats an identifier.", graph.macros.items[0].description);
+    try std.testing.expectEqual(@as(usize, 2), graph.macros.items[0].arguments.items.len);
+    try std.testing.expectEqualStrings("column_name", graph.macros.items[0].arguments.items[0].name);
+    try std.testing.expectEqualStrings("string", graph.macros.items[0].arguments.items[0].type);
+    try std.testing.expectEqualStrings("Identifier expression.", graph.macros.items[0].arguments.items[0].description);
+    try std.testing.expectEqualStrings("quote", graph.macros.items[0].arguments.items[1].name);
+    try std.testing.expectEqualStrings("bool", graph.macros.items[0].arguments.items[1].type);
+}
+
+test "applyMacroProperties records unmatched macro properties" {
+    var arena = std.heap.ArenaAllocator.init(std.testing.allocator);
+    defer arena.deinit();
+    const allocator = arena.allocator();
+
+    var graph = Graph{ .allocator = allocator, .project_name = "demo" };
+    defer graph.deinit();
+
+    try graph.macro_properties.append(allocator, .{
+        .package_name = "demo",
+        .name = "missing_macro",
+        .patch_path = "macros/schema.yml",
+    });
+
+    try applyMacroProperties(&graph);
+
+    try std.testing.expectEqual(@as(usize, 1), graph.unmatched_macro_properties.items.len);
+    try std.testing.expectEqualStrings("missing_macro", graph.unmatched_macro_properties.items[0].name);
+    try std.testing.expectEqualStrings("macros/schema.yml", graph.unmatched_macro_properties.items[0].patch_path);
+}
+
 test "refDepFromValue parses relationship target refs and raw model names" {
     var arena = std.heap.ArenaAllocator.init(std.testing.allocator);
     defer arena.deinit();