feat: Mark module sources as read (#5963)

* feat: Mark module sources as read

* feat: Consolidating into one `internal/glob` package

fix: Sticking to gobaws

* chore: Integrating `vfs`

docs: Docs cleanup

* fix: Addressing PR review

Author: yhakbar

docs/bun.lock

@@ -8,6 +8,7 @@ sidebar:
 
 import { Aside } from '@astrojs/starlight/components';
 import FileTree from '@components/vendored/starlight/FileTree.astro';
+import Since from '@components/Since.astro';
 
 Match units and stacks by their configuration attributes.
 
@@ -104,6 +105,14 @@ Due to how Terragrunt parses configurations during `run --all`, functions will o
 Reading a file directly in the `inputs` attribute will not mark the file as read, as the `inputs` attribute is not parsed until after the queue has already been populated to support rendering of dependency outputs, which are only available after dependencies have been run.
 </Aside>
 
+<Since version="1.0.3">
+
+To mark many files at once, use [`mark_glob_as_read`](/reference/hcl/functions/#mark_glob_as_read), which accepts a glob (with `**` support) and marks every matching file.
+
+If a unit's `terraform` block points at a local module source, enable the [`mark-many-as-read`](/reference/experiments) experiment to have Terragrunt automatically mark that module's `*.tf`, `*.tf.json`, `*.hcl`, `*.tofu`, and `*.tofu.json` files as read. Reading-based filters will then cascade local module changes to every unit that consumes the module. The same experiment also enables the [`mark_glob_as_read`](/reference/hcl/functions/#mark_glob_as_read) HCL function.
+
+</Since>
+
 ## Source-Based Expressions
 
 Match units and stacks by their Terraform source URL or path specified in the `terraform` block of `terragrunt.hcl` files.

docs/src/content/docs/03-features/08-filter/04-attributes.mdx

@@ -6,7 +6,10 @@ sidebar:
   order: 4
 ---
 
+import { Aside } from '@astrojs/starlight/components';
 import FileTree from '@components/vendored/starlight/FileTree.astro';
+import Since from '@components/Since.astro';
+import Before from '@components/Before.astro';
 
 Terragrunt allows you to use built-in functions anywhere in `terragrunt.hcl`, just like OpenTofu/Terraform\!
 
@@ -1042,6 +1045,62 @@ The same technique can be used to mark a file as read when a file is read using
 if they are used in the `locals` block. Reading a file directly in the `inputs` block will not mark the file as read, as the `inputs`
 block is not evaluated until *after* the queue has been populated with units to run.
 
+## mark_glob_as_read
+
+<Before version="1.0.3">
+
+<Aside type="tip">
+Coming soon!
+</Aside>
+
+</Before>
+
+<Since version="1.0.3">
+
+<Aside type="caution">
+`mark_glob_as_read` requires the [`mark-many-as-read`](/reference/experiments) experiment. Calling it without the experiment enabled returns an error.
+</Aside>
+
+<Aside type="caution">
+`mark_glob_as_read` writes to the same read tracker as [`mark_as_read`](#mark_as_read), so the same evaluation-order caveat applies: during a `run --all`, the queue is populated before `inputs` is evaluated, so a call placed in `inputs` (or any other later-evaluated block) will not influence queue construction. Call `mark_glob_as_read` from `locals` to ensure matched files are considered when the queue is built.
+</Aside>
+
+`mark_glob_as_read(pattern)` expands the given glob and marks every matching file as read. It returns the list of absolute paths that matched, so it can be composed with other expressions.
+
+Pattern syntax follows [`gobwas/glob`](https://github.com/gobwas/glob), the same matcher used elsewhere in Terragrunt. `/` is the path separator, `*` matches within a single segment, `**` matches any sequence of characters including separators, `?` matches any single non-separator character, `[abc]` matches a character class, and `{a,b}` matches any of the listed alternatives. A backslash escapes the following metacharacter.
+
+Relative patterns are resolved against the current working directory, the same way `mark_as_read` resolves a relative file path.
+
+<Aside type="caution">
+Glob patterns must use forward slashes (`/`) as the path separator, even on Windows. The matcher treats `\` as an escape character, not a path separator, so a backslash-separated pattern will not match any files.
+</Aside>
+
+<Aside type="caution">
+`**` only collapses the separators around it when the adjacent segments are literals. `config/**/*.yaml` matches files at least one directory below `config/`, but not files directly inside `config/`, because the trailing segment `*.yaml` is a wildcard. Use brace alternation to match at any depth:
+
+```hcl
+mark_glob_as_read("${get_terragrunt_dir()}/config/{*.yaml,**/*.yaml}")
+```
+</Aside>
+
+```hcl
+# terragrunt.hcl
+
+locals {
+  configs = mark_glob_as_read("${get_terragrunt_dir()}/config/{*.yaml,**/*.yaml}")
+}
+
+inputs = {
+  configs = [for f in local.configs : yamldecode(file(f))]
+}
+```
+
+A typical use case is a unit that consumes a directory of configuration files indirectly (for example, through `run_cmd`, `templatefile`, or a shared helper module) and wants changes to any of them to trigger the unit through [reading-based filter expressions](/features/filter/attributes#reading-based-expressions) such as `--filter 'reading=./config/**'`.
+
+If the pattern matches nothing, the function returns an empty list without error.
+
+</Since>
+
 ## constraint_check
 
 `constraint_check(version, constraint)` checks if a given version satisfies a given constraint.

docs/src/content/docs/04-reference/01-hcl/04-functions.mdx

@@ -0,0 +1,31 @@
+---
+version: "v1.0.3"
+category: "experiments-added"
+---
+
+#### `mark-many-as-read` — Mark many files as read in one step
+
+Enable the new [`mark-many-as-read`](/reference/experiments) experiment to turn on two behaviors that each mark many files as read in a single call: automatic marking of files inside a local `terraform { source = "..." }` block, and the new `mark_glob_as_read` HCL function.
+
+With the experiment on, a unit like this:
+
+```hcl
+# live/unit/terragrunt.hcl
+terraform {
+  source = "../../modules/service"
+}
+```
+
+records every `*.tf`, `*.tf.json`, `*.hcl`, `*.tofu`, and `*.tofu.json` file found under `../../modules/service` (recursively) as read for the unit. Non-source files such as `README.md` are skipped. A [reading-based filter expression](/features/filter/attributes#reading-based-expressions) such as `--filter 'reading=../../modules/service/**'` then matches every unit that points at the module, so a change to any file in the module cascades to its consumers.
+
+The same experiment also enables a new HCL function, `mark_glob_as_read(pattern)`, which expands a glob using the same [`gobwas/glob`](https://github.com/gobwas/glob) syntax as filter expressions and marks every matching file as read. It returns the list of absolute paths that matched, so it composes with other expressions:
+
+```hcl
+locals {
+  configs = mark_glob_as_read("${get_terragrunt_dir()}/config/{*.yaml,**/*.yaml}")
+}
+```
+
+`**` only collapses the surrounding separators when the adjacent segments are literals, so match-at-any-depth with a wildcard trailing segment is written as `{*.yaml,**/*.yaml}`. See the [HCL reference](/reference/hcl/functions/#mark_glob_as_read) for full pattern syntax.
+
+This is useful when a unit reads a collection of files indirectly (for example, via `run_cmd` or `templatefile`) and you want changes to any of them to trigger the unit through reading-based filters. Calling `mark_glob_as_read` without the experiment enabled returns an error.

docs/src/data/changelog/v1.0.3/mark-many-as-read.mdx

@@ -0,0 +1,33 @@
+---
+name: mark-many-as-read
+status: active
+since: "1.0.3"
+---
+
+Mark many files as read in one step, so reading-based filter expressions cascade changes from shared files to the units that consume them.
+
+### `mark-many-as-read` - What it does
+
+Enabling the experiment activates two behaviors:
+
+1. When a unit's `terraform` block points at a local module source, Terragrunt walks that directory and records every `*.tf`, `*.tf.json`, `*.hcl`, `*.tofu`, and `*.tofu.json` file as read for the unit. Non-source files such as `README.md` are skipped. Remote sources (Git, registry, S3, etc.) are not walked.
+2. The `mark_glob_as_read(pattern)` HCL function becomes available. It expands a glob using [`gobwas/glob`](https://github.com/gobwas/glob) syntax and marks every matching file as read, returning the list of absolute paths that matched. Without the experiment enabled, calling the function returns an error. See the [HCL reference](/reference/hcl/functions/#mark_glob_as_read) for pattern syntax and examples.
+
+Both behaviors feed the same reading tracker that powers the [`reading=` filter attribute](/features/filter/attributes#reading-based-expressions), so a change to a local module file or a globbed configuration file is picked up by `--filter 'reading=<path>'` and matches every unit that reads it.
+
+```bash
+terragrunt run --all --experiment mark-many-as-read -- plan
+```
+
+### `mark-many-as-read` - How to provide feedback
+
+Provide your feedback on the [GitHub Discussions](https://github.com/gruntwork-io/terragrunt/discussions) page.
+
+### `mark-many-as-read` - Criteria for stabilization
+
+To transition the `mark-many-as-read` feature to a stable release, the following must be addressed:
+
+- [ ] Confirm local module walking handles nested modules sensibly across macOS, Linux, and Windows.
+- [ ] Confirm glob semantics match user expectations for common patterns, especially `**` with a wildcard trailing segment.
+- [ ] Positive feedback from users relying on reading-based filters in production pipelines.
+- [ ] Integration tests covering the interaction between module walking, `mark_glob_as_read`, and `--filter 'reading=...'`.

docs/src/data/experiments/mark-many-as-read.mdx

@@ -51,6 +51,11 @@ const (
 	StackDependencies = "stack-dependencies"
 	// CatalogRedesign is the experiment that enables the redesigned catalog experience.
 	CatalogRedesign = "catalog-redesign"
+	// MarkManyAsRead enables behaviors that mark many files as read in one
+	// step: automatic marking of files inside a local terraform module source
+	// (so reading-based filter expressions detect changes to the module) and
+	// the mark_glob_as_read HCL function.
+	MarkManyAsRead = "mark-many-as-read"
 )
 
 const (
@@ -115,6 +120,9 @@ func NewExperiments() Experiments {
 		{
 			Name: CatalogRedesign,
 		},
+		{
+			Name: MarkManyAsRead,
+		},
 	}
 }
 

internal/experiment/experiment.go

@@ -4,8 +4,8 @@ import (
 	"path/filepath"
 	"strconv"
 
-	"github.com/gobwas/glob"
 	"github.com/gruntwork-io/terragrunt/internal/component"
+	"github.com/gruntwork-io/terragrunt/internal/glob"
 )
 
 // Expression is the interface that all AST nodes must implement.
@@ -31,15 +31,15 @@ type Expressions []Expression
 
 // PathExpression represents a path or glob filter (e.g., "./path/**/*" or "/absolute/path").
 type PathExpression struct {
-	compiledGlob glob.Glob
+	compiledGlob glob.Matcher
 	Value        string
 }
 
 // NewPathFilter creates a new PathFilter with eager glob compilation.
 func NewPathFilter(value string) (*PathExpression, error) {
 	pattern := filepath.Clean(filepath.ToSlash(value))
 
-	compiled, err := glob.Compile(pattern, '/')
+	compiled, err := glob.Compile(pattern)
 	if err != nil {
 		return nil, err
 	}
@@ -48,7 +48,7 @@ func NewPathFilter(value string) (*PathExpression, error) {
 }
 
 // Glob returns the pre-compiled glob pattern.
-func (p *PathExpression) Glob() glob.Glob {
+func (p *PathExpression) Glob() glob.Matcher {
 	return p.compiledGlob
 }
 
@@ -61,7 +61,7 @@ func (p *PathExpression) Negated() Expression                   { return NewPref
 
 // AttributeExpression represents a key-value attribute filter (e.g., "name=my-app").
 type AttributeExpression struct {
-	compiledGlob glob.Glob
+	compiledGlob glob.Matcher
 	Key          string
 	Value        string
 }
@@ -78,7 +78,7 @@ func NewAttributeExpression(key string, value string) (*AttributeExpression, err
 			pattern = filepath.Clean(filepath.ToSlash(pattern))
 		}
 
-		compiled, err := glob.Compile(pattern, '/')
+		compiled, err := glob.Compile(pattern)
 		if err != nil {
 			return nil, err
 		}
@@ -96,7 +96,7 @@ func NewTypeExpression(kind component.Kind) *AttributeExpression {
 }
 
 // Glob returns the pre-compiled glob pattern.
-func (a *AttributeExpression) Glob() glob.Glob {
+func (a *AttributeExpression) Glob() glob.Matcher {
 	return a.compiledGlob
 }
 

internal/filter/ast.go

@@ -210,7 +210,7 @@
 // ## Evaluator
 //
 // The evaluator (evaluator.go) walks the AST and applies the filter logic:
-//   - PathFilter: Uses glob matching (github.com/gobwas/glob) with eager compilation
+//   - PathFilter: Uses glob matching (internal/glob Compile, backed by gobwas) with eager compilation
 //     and caching via sync.Once for performance
 //   - AttributeFilter: Matches attributes by key-value pairs:
 //   - name: Matches filepath.Base(component.Path)