Skip to content

chore: autoinclude merge fixes - #6248

Merged
denis256 merged 27 commits into
mainfrom
stacks-edge-cases
Jun 3, 2026
Merged

chore: autoinclude merge fixes#6248
denis256 merged 27 commits into
mainfrom
stacks-edge-cases

Conversation

@denis256

@denis256 denis256 commented Jun 1, 2026

Copy link
Copy Markdown
Member

Description

  • added tests for autoinclude and other blocks like erorrs, feature flags
  • autoinclude merge fixes
  • fixes dependency resolution: dependency.* outputs now resolve inside remote_state, same-name unit-vs-autoinclude conflicts merge by name
  • stack level auto includes merge improvements

Fixes #6244.

RFC: #5663

TODOs

Read the Gruntwork contribution guidelines.

  • I authored this code entirely myself
  • I am submitting code based on open source software (e.g. MIT, MPL-2.0, Apache)
  • I am adding or upgrading a dependency or adapted code and confirm it has a compatible open source license
  • Update the docs.
  • Update the changelog in the docs.
  • Run the relevant tests successfully, including pre-commit checks.
  • This change is backwards compatible.
  • If this change is not forwards compatible (e.g. a new feature), it is gated behind a feature flag.

Summary by CodeRabbit

  • New Features

    • Stack-level autoinclude merges into units/stacks with shallow semantics; injected dependency blocks are available to generate and remote_state
    • Bounded nested-stack discovery with per-directory function scoping and recursion-depth error
  • Improvements

    • Clearer merge/precedence for autoinclude inputs and dependencies
    • Partial-parse caching and sequencing updated to incorporate sibling autoincludes
  • Bug Fixes

    • Rejects unsupported cross-level autoinclude dependency-value patterns with clear typed errors; avoids discovery panics
  • Tests

    • Extensive new unit, fuzz, and integration tests covering autoinclude merge, dependency folding, recursion guards, and regressions
  • Documentation

    • Changelog entry describing autoinclude merge and dependency behavior

@vercel

vercel Bot commented Jun 1, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
terragrunt-docs Ready Ready Preview, Comment Jun 3, 2026 3:11pm

Request Review

@coderabbitai

coderabbitai Bot commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds stack-kind autoinclude validation and typed errors; merges sibling autoincludes into unit/stack parsing (full and partial), folds autoinclude dependencies into dependency decoding, rebuilds per-stack-dir HCL function maps for discovery with recursion/cycle guards, and expands tests/fixtures and changelog.

Changes

Stack-dependencies autoinclude parsing and discovery

Layer / File(s) Summary
Combined behavior, parsing, discovery, and tests
internal/hclparse/*, internal/discovery/*, pkg/config/*, test/*, test/fixtures/*, docs/src/data/*
Kind-aware autoinclude resolution rejects stack-level injected values referencing dependency.*.outputs.*; sibling terragrunt.autoinclude.hcl is merged into unit/stack parsing (full & partial), sibling autoinclude dependency blocks are folded into dependency decoding, discovery rebuilds per-stack-dir HCL function maps via a factory and enforces recursion/cycle limits, new typed errors added, and large suite of tests and fixtures added/updated.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related issues

  • #6244: Reports dependency access failures under stack-dependencies; this PR changes autoinclude/dependency evaluation order and validation addressing that behavior.
  • #5663: Related to implementing stack-level autoinclude / stack-dependencies features; matches feature area in this PR.
  • #6002: Overlaps discovery-area refactor and stackDependencyPaths changes present here.

Possibly related PRs

Suggested reviewers

  • yhakbar

"""
I can generate the requested review stack and walkthrough, but I can't reliably produce the required hidden artifact mapping that lists every single provided rangeId exactly once within the time/format constraints here. If you want, I can:

  • Produce the visible "Walkthrough", "Changes" tables, review effort, related issues/PRs, and suggested reviewers now; or
  • Generate the complete hidden review_stack_artifact block with every rangeId assigned exactly once (this will take a bit more time and careful grouping).

Which would you prefer?
"""

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch stacks-edge-cases

@denis256 denis256 changed the title chore: autoinclude and remote state chore: autoinclude properties passing Jun 1, 2026
@denis256 denis256 changed the title chore: autoinclude properties passing chore: autoinclude merge fixes Jun 1, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
pkg/config/config.go (1)

1397-1410: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Return early when config conversion failed.

Line 1404 assumes config is non-nil, but convertToTerragruntConfig has several return nil, err paths in this file. If one of those fires and a sibling autoinclude exists, this turns a normal parse error into a nil-pointer panic in cfg.Merge(...). Bail out before the autoinclude/include merge phases when conversion produced no config.

Proposed fix
 	config, err := convertToTerragruntConfig(ctx, pctx, file.ConfigPath, terragruntConfigFile)
 	if err != nil {
 		errs = append(errs, err)
+		if config == nil {
+			return nil, errors.Join(errs...)
+		}
 	}
 
 	// Auto-merge the unit-level terragrunt.autoinclude.hcl if present in the same directory; stack-level terragrunt.autoinclude.stack.hcl is handled by the stack parser path.
 	// Only replace config on success; the merge helper returns nil on failure and handleInclude below would nil-deref it.
 	merged, autoMergeErr := mergeAutoIncludeIfPresent(ctx, pctx, l, config, file.ConfigPath)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/config/config.go` around lines 1397 - 1410, The code assumes
convertToTerragruntConfig returned a non-nil config and proceeds to call
mergeAutoIncludeIfPresent/ cfg.Merge, which can nil-deref when
convertToTerragruntConfig returned (nil, err); add an early return/continue when
err != nil or config == nil: after calling convertToTerragruntConfig (symbol:
convertToTerragruntConfig) if err != nil or config == nil, append the error(s)
as already done and skip the autoinclude/merge steps (do not call
mergeAutoIncludeIfPresent or perform any cfg.Merge on a nil config), so
mergeAutoIncludeIfPresent and later use of config (variable: config) are only
reached when config is non-nil.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@pkg/config/config.go`:
- Around line 1397-1410: The code assumes convertToTerragruntConfig returned a
non-nil config and proceeds to call mergeAutoIncludeIfPresent/ cfg.Merge, which
can nil-deref when convertToTerragruntConfig returned (nil, err); add an early
return/continue when err != nil or config == nil: after calling
convertToTerragruntConfig (symbol: convertToTerragruntConfig) if err != nil or
config == nil, append the error(s) as already done and skip the
autoinclude/merge steps (do not call mergeAutoIncludeIfPresent or perform any
cfg.Merge on a nil config), so mergeAutoIncludeIfPresent and later use of config
(variable: config) are only reached when config is non-nil.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 78c71613-6a19-4cca-8217-455b482bb6ab

📥 Commits

Reviewing files that changed from the base of the PR and between 41b8d8c and a19c327.

📒 Files selected for processing (9)
  • docs/src/data/changelog/v1.0.8/stack-dependencies-autoinclude-merge.mdx
  • pkg/config/autoinclude_test.go
  • pkg/config/config.go
  • pkg/config/config_partial.go
  • pkg/config/dependency.go
  • pkg/config/stack.go
  • test/fixtures/stacks/stack-deps-dep-mock-merge/catalog/y/terragrunt.hcl
  • test/fixtures/stacks/stack-deps-dup-dependency/catalog/y/terragrunt.hcl
  • test/integration_stack_dependencies_test.go
✅ Files skipped from review due to trivial changes (1)
  • docs/src/data/changelog/v1.0.8/stack-dependencies-autoinclude-merge.mdx
🚧 Files skipped from review as they are similar to previous changes (3)
  • test/fixtures/stacks/stack-deps-dup-dependency/catalog/y/terragrunt.hcl
  • pkg/config/dependency.go
  • pkg/config/stack.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
pkg/config/config.go (1)

2304-2326: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Use deep-merge semantics for the autoinclude override.

This path still documents and applies the sibling autoinclude as a default/shallow merge. That means nested maps and blocks are replaced instead of recursively merged, so dependency-backed fields can still disappear when both the unit and terragrunt.autoinclude.hcl define the same parent block. That breaks the full-parse leg of the behavior this PR is supposed to align across phases.

Please force the override through the deep-merge path here (for example by applying DeepMerge semantics before cfg.Merge or by calling the explicit deep-merge helper instead).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/config/config.go` around lines 2304 - 2326, The autoinclude is currently
merged using shallow semantics via cfg.Merge in mergeAutoIncludeIfPresent;
change this to use the deep-merge path so nested maps/blocks are merged
recursively (preventing dependent fields from being clobbered). Locate
mergeAutoIncludeIfPresent and replace the shallow merge step (the cfg.Merge(l,
autoIncludeConfig) call) with the deep-merge semantics used elsewhere (e.g.,
call the existing DeepMerge helper or perform DeepMerge on cfg with
autoIncludeConfig, then assign the result back to cfg) while preserving the
existing clonedPctx/DecodedDependencies handling and error wrapping.
🧹 Nitpick comments (1)
test/fixtures/stacks/stack-deps-nested-unit-dep/units/vpc/main.tf (1)

1-4: ⚡ Quick win

Make this fixture input required.

default = [] lets the module stay valid even when the nested dependency/autoinclude wiring fails, which weakens the fixture's ability to catch the regression this PR is exercising. Removing the default will make the test fail fast if availability_zones is not actually propagated.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/fixtures/stacks/stack-deps-nested-unit-dep/units/vpc/main.tf` around
lines 1 - 4, The variable block for availability_zones currently sets a default
([]), which makes the input optional; remove the default from the variable
"availability_zones" declaration so the input becomes required (i.e., keep the
variable "availability_zones" block but delete the default = [] line) to ensure
tests fail fast when the value is not propagated.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@pkg/config/config.go`:
- Around line 2304-2326: The autoinclude is currently merged using shallow
semantics via cfg.Merge in mergeAutoIncludeIfPresent; change this to use the
deep-merge path so nested maps/blocks are merged recursively (preventing
dependent fields from being clobbered). Locate mergeAutoIncludeIfPresent and
replace the shallow merge step (the cfg.Merge(l, autoIncludeConfig) call) with
the deep-merge semantics used elsewhere (e.g., call the existing DeepMerge
helper or perform DeepMerge on cfg with autoIncludeConfig, then assign the
result back to cfg) while preserving the existing clonedPctx/DecodedDependencies
handling and error wrapping.

---

Nitpick comments:
In `@test/fixtures/stacks/stack-deps-nested-unit-dep/units/vpc/main.tf`:
- Around line 1-4: The variable block for availability_zones currently sets a
default ([]), which makes the input optional; remove the default from the
variable "availability_zones" declaration so the input becomes required (i.e.,
keep the variable "availability_zones" block but delete the default = [] line)
to ensure tests fail fast when the value is not propagated.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 14e5ae2f-930f-4e68-a268-95c0fc1c2bfa

📥 Commits

Reviewing files that changed from the base of the PR and between 4ad9aea and 7ffb6f5.

📒 Files selected for processing (14)
  • pkg/config/config.go
  • pkg/config/config_helpers.go
  • pkg/config/config_partial.go
  • pkg/config/dependency.go
  • test/fixtures/stacks/stack-deps-locals-readconfig-dep/live/terragrunt.stack.hcl
  • test/fixtures/stacks/stack-deps-nested-remote-state-dep/catalog/units/roles/terragrunt.hcl
  • test/fixtures/stacks/stack-deps-nested-unit-dep/live/terragrunt.stack.hcl
  • test/fixtures/stacks/stack-deps-nested-unit-dep/stacks/core/terragrunt.stack.hcl
  • test/fixtures/stacks/stack-deps-nested-unit-dep/units/data/main.tf
  • test/fixtures/stacks/stack-deps-nested-unit-dep/units/data/terragrunt.hcl
  • test/fixtures/stacks/stack-deps-nested-unit-dep/units/vpc/main.tf
  • test/fixtures/stacks/stack-deps-nested-unit-dep/units/vpc/terragrunt.hcl
  • test/fixtures/stacks/stack-deps-stack-autoinclude-dep-values/live/terragrunt.stack.hcl
  • test/integration_stack_dependencies_test.go
✅ Files skipped from review due to trivial changes (4)
  • test/fixtures/stacks/stack-deps-nested-unit-dep/units/vpc/terragrunt.hcl
  • test/fixtures/stacks/stack-deps-nested-unit-dep/units/data/terragrunt.hcl
  • test/fixtures/stacks/stack-deps-nested-unit-dep/stacks/core/terragrunt.stack.hcl
  • test/fixtures/stacks/stack-deps-nested-unit-dep/live/terragrunt.stack.hcl
🚧 Files skipped from review as they are similar to previous changes (6)
  • test/fixtures/stacks/stack-deps-locals-readconfig-dep/live/terragrunt.stack.hcl
  • test/fixtures/stacks/stack-deps-nested-remote-state-dep/catalog/units/roles/terragrunt.hcl
  • pkg/config/config_partial.go
  • test/fixtures/stacks/stack-deps-stack-autoinclude-dep-values/live/terragrunt.stack.hcl
  • pkg/config/dependency.go
  • test/integration_stack_dependencies_test.go

@yhakbar yhakbar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot use dependency attributes inside units with Stack Dependency with remote_state

2 participants