Skip to content

Commit cf21fba

Browse files
authored
docs: Documenting --filter Git support (#5108)
* docs: Documenting Git-based expressions * docs: Adding comprehensive list of supported attributes
1 parent b6ce13a commit cf21fba

1 file changed

Lines changed: 154 additions & 63 deletions

File tree

docs-starlight/src/content/docs/03-features/18-filter.mdx

Lines changed: 154 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -69,11 +69,22 @@ For the following file tree:
6969
- terragrunt.hcl
7070
</FileTree>
7171

72-
## Filter Types
72+
## Filter Expressions
7373

74-
There are several different types of filters, and particular ways in which they can be combined to achieve different results. You can learn more about that below.
74+
There are several different types of filter expressions, and particular ways in which they can be combined to achieve different results. You can learn more about that below.
7575

76-
### Name-Based Filtering
76+
| Filter Type | Description |
77+
|-------------|-------------|
78+
| Name-Based | Match units and stacks by their name. |
79+
| Path-Based | Match units and stacks by their file system path. |
80+
| Attribute-Based | Match units and stacks by their configuration attributes. |
81+
| Negation | Exclude units and stacks using the `!` prefix. |
82+
| Intersection | Use the `|` operator to refine results. |
83+
| Union | Combine filter results using multiple `--filter` flags. |
84+
| Graph-Based | Filter units based on their dependency relationships using graph traversal operators. |
85+
| Git-Based | Filter units and stacks based on Git diffs using Git expressions. |
86+
87+
### Name-Based Expressions
7788

7889
Match units and stacks by their name. This is the simplest form of filtering.
7990

@@ -99,7 +110,7 @@ terragrunt find --filter 'app*'
99110
Note that `app1` and `app2` were selected _within_ the `apps` directory. Filtering on names will match _any_ unit/stack that has a name that matches the filter.
100111
</Aside>
101112

102-
### Path-Based Filtering
113+
### Path-Based Expressions
103114

104115
Match units and stacks by their file system path.
105116

@@ -143,7 +154,7 @@ Note that globs used in path-based filtering will not recursively match nested d
143154
(That's why `./envs/stage/**` is used above)
144155
</Aside>
145156

146-
### Attribute-Based Filtering
157+
### Attribute-Based Expressions
147158

148159
Match units and stacks by their configuration attributes.
149160

@@ -172,7 +183,17 @@ terragrunt find --filter 'name=stack*'
172183
- terragrunt.hcl
173184
</FileTree>
174185

175-
### Source-Based Filtering
186+
The following are the attributes supported for attribute-based filtering:
187+
188+
| Attribute | Description |
189+
|-----------|-------------|
190+
| name | Match units and stacks by their name. |
191+
| type | Match units and stacks by their type. |
192+
| external | Match units and stacks by their external dependency status. |
193+
| reading | Match units and stacks by the files they read. |
194+
| source | Match units and stacks by their Terraform source URL or path specified in the `terraform` block of `terragrunt.hcl` files. |
195+
196+
### Source-Based Expressions
176197

177198
Match units and stacks by their Terraform source URL or path specified in the `terraform` block of `terragrunt.hcl` files.
178199

@@ -287,6 +308,67 @@ terragrunt find --filter './dev/** | type=unit | !name=unit1'
287308
- terragrunt.stack.hcl
288309
</FileTree>
289310

311+
### Union (Multiple Filters)
312+
313+
Specify multiple `--filter` flags to combine results using OR logic.
314+
315+
```bash
316+
# Find components named 'unit1' OR 'stack1'
317+
terragrunt find --filter unit1 --filter stack1
318+
319+
# Find components in ./envs/prod/* OR ./envs/stage/*
320+
terragrunt find --filter './envs/prod/*' --filter './envs/stage/*'
321+
322+
# Find components named 'stack2' _except_ those in ./envs/prod/* and ./envs/stage/*
323+
terragrunt find --filter stack2 --filter '!./envs/prod/**' --filter '!./envs/stage/**'
324+
```
325+
326+
<FileTree>
327+
- envs
328+
- prod
329+
- **unit1** \<-- Matched by the first filter _and_ the second filter
330+
- terragrunt.hcl
331+
- **unit2** \<-- Matched by the second filter
332+
- terragrunt.hcl
333+
- **stack1** \<-- Matched by the first filter _and_ the second filter
334+
- terragrunt.stack.hcl
335+
- **stack2** \<-- Matched by the second filter
336+
- terragrunt.stack.hcl
337+
- stage
338+
- **unit1** \<-- Matched by the first filter _and_ the second filter
339+
- terragrunt.hcl
340+
- **unit2** \<-- Matched by the second filter
341+
- terragrunt.hcl
342+
- **stack1** \<-- Matched by the first filter _and_ the second filter
343+
- terragrunt.stack.hcl
344+
- **stack2** \<-- Matched by the second filter
345+
- terragrunt.stack.hcl
346+
- dev
347+
- **unit1** \<-- Matched by the first filter
348+
- terragrunt.hcl
349+
- unit2
350+
- terragrunt.hcl
351+
- **stack1** \<-- Matched by the first filter
352+
- terragrunt.stack.hcl
353+
- **stack2** \<-- Matched by the third filter
354+
- terragrunt.stack.hcl
355+
</FileTree>
356+
357+
<Aside type="caution" title="Unions of negated filters">
358+
359+
When a filter query starts with a negation (`!`), the result is applied after _all_ positive filters have been applied.
360+
361+
This means that if you have a filter query like this:
362+
```bash
363+
terragrunt find --filter '!type=unit' --filter 'name=unit1'
364+
```
365+
366+
The result will be the components that are not units _and_ are named `unit1`.
367+
368+
As a result, you should be able to expect any negative filter to take effect, regardless of how other positive filters may result in the addition of results.
369+
370+
</Aside>
371+
290372
### Graph-Based Filtering
291373

292374
Filter units and stacks based on their dependency relationships using graph traversal operators. This allows you to find components that depend on a target, or components that a target depends on.
@@ -406,93 +488,102 @@ terragrunt find --filter './prod/** | ...vpc'
406488
Graph expressions require dependency/dependent discovery to work correctly. When using graph expressions, Terragrunt automatically discovers dependency relationships between components to enable graph traversal. This may add some overhead compared to simple name or path filters.
407489
</Aside>
408490

409-
### Union (Multiple Filters)
491+
### Git-Based Filtering
410492

411-
Specify multiple `--filter` flags to combine results using OR logic.
493+
Match units and stacks based on changes between Git references. This is useful for targeting infrastructure that has been modified, added, or removed between commits, branches, or tags.
412494

413495
```bash
414-
# Find components named 'unit1' OR 'stack1'
415-
terragrunt find --filter unit1 --filter stack1
496+
# Compare between two references
497+
terragrunt find --filter '[main...HEAD]'
416498

417-
# Find components in ./envs/prod/* OR ./envs/stage/*
418-
terragrunt find --filter './envs/prod/*' --filter './envs/stage/*'
499+
# Shorthand: compare reference to HEAD
500+
terragrunt find --filter '[main]'
419501

420-
# Find components named 'stack2' _except_ those in ./envs/prod/* and ./envs/stage/*
421-
terragrunt find --filter stack2 --filter '!./envs/prod/**' --filter '!./envs/stage/**'
502+
# Compare between specific commits
503+
terragrunt find --filter '[abc123...def456]'
504+
505+
# Compare between tags
506+
terragrunt find --filter '[v1.0.0...v2.0.0]'
507+
508+
# Compare using relative references
509+
terragrunt find --filter '[HEAD~1...HEAD]'
510+
511+
# Compare between branches
512+
terragrunt find --filter '[feature-branch...main]'
422513
```
423514

424515
<FileTree>
425-
- envs
426-
- prod
427-
- **unit1** \<-- Matched by the first filter _and_ the second filter
428-
- terragrunt.hcl
429-
- **unit2** \<-- Matched by the second filter
430-
- terragrunt.hcl
431-
- **stack1** \<-- Matched by the first filter _and_ the second filter
432-
- terragrunt.stack.hcl
433-
- **stack2** \<-- Matched by the second filter
434-
- terragrunt.stack.hcl
435-
- stage
436-
- **unit1** \<-- Matched by the first filter _and_ the second filter
437-
- terragrunt.hcl
438-
- **unit2** \<-- Matched by the second filter
439-
- terragrunt.hcl
440-
- **stack1** \<-- Matched by the first filter _and_ the second filter
441-
- terragrunt.stack.hcl
442-
- **stack2** \<-- Matched by the second filter
443-
- terragrunt.stack.hcl
444-
- dev
445-
- **unit1** \<-- Matched by the first filter
446-
- terragrunt.hcl
447-
- unit2
448-
- terragrunt.hcl
449-
- **stack1** \<-- Matched by the first filter
450-
- terragrunt.stack.hcl
451-
- **stack2** \<-- Matched by the third filter
452-
- terragrunt.stack.hcl
516+
- .
517+
- **modified-unit** \<-- Matched by [main...HEAD] (terragrunt.hcl was modified)
518+
- terragrunt.hcl (modified)
519+
- **new-unit** \<-- Matched by [main...HEAD] (terragrunt.hcl was added)
520+
- terragrunt.hcl (added)
521+
- **removed-unit** \<-- Matched by [main...HEAD] (terragrunt.hcl was removed)
522+
- (directory removed)
523+
- unchanged-unit
524+
- terragrunt.hcl (unchanged)
453525
</FileTree>
454526

455-
<Aside type="caution" title="Unions of negated filters">
527+
<Aside type="note">
528+
When using Git-based filtering and the `run` command, you are required to use one of the `plan` or `apply` commands, and not the `-destroy` flag.
456529

457-
When a filter query starts with a negation (`!`), the result is applied after _all_ positive filters have been applied.
530+
This is because whether or not a unit will be destroyed is determined by logic relevant to inspecting changes in Git.
531+
532+
When units are added or modified between two Git references, they will be be planned or applied. When the units are removed between two Git references, they will be planned for destruction (with `plan -destroy) or destroyed (with `apply -destroy`).
533+
534+
In the scenario above, running the following:
458535

459-
This means that if you have a filter query like this:
460536
```bash
461-
terragrunt find --filter '!type=unit' --filter 'name=unit1'
537+
terragrunt run --filter '[main...HEAD]' plan
462538
```
463539

464-
The result will be the components that are not units _and_ are named `unit1`.
540+
Will result in the following:
465541

466-
As a result, you should be able to expect any negative filter to take effect, regardless of how other positive filters may result in the addition of results.
542+
- `modified-unit` will be planned (`tofu plan`)
543+
- `new-unit` will be planned (`tofu plan`)
544+
- `removed-unit` will be planned for destruction (`tofu plan -destroy`)
545+
- `unchanged-unit` will be ignored
467546

547+
Note that you will also receive a warning when doing this that you must provide the `--filter-allow-destroy` flag to allow destruction to occur on `apply`. This is a safeguard to prevent accidental destruction of infrastructure.
468548
</Aside>
469549

470-
## Usage with Commands
550+
<Aside type="tip" title="How it works">
471551

472-
The following commands all support the `--filter` flag, and use it to filter results in the same way:
552+
When evaluating a Git-based filter, Terragrunt will first generate a worktree for every reference that needs to be evaluated, and assess the diffs between Git references.
473553

474-
- [x] [find](/docs/reference/cli/commands/find)
475-
- [x] [list](/docs/reference/cli/commands/list)
476-
- [x] [run](/docs/reference/cli/commands/run)
477-
- [x] [hcl fmt](/docs/reference/cli/commands/hcl/fmt)
478-
- [x] [hcl validate](/docs/reference/cli/commands/hcl/validate)
554+
e.g. For a filter like `[main...HEAD]`, Terragrunt will generate a worktree for `main` and one for `HEAD` in temporary directories, and use `git diff` to assess the diffs between the two references.
479555

480-
This flag is intended to be a flexible way to target specific infrastructure that allows you to dry-run infrastructure targeting using discovery commands (like `find` and `list`) before running a command that actually affects infrastructure (like `run`).
556+
Then, for any unit that is discovered within those worktrees, Terragrunt will enqueue that unit for a run in the run queue _in the worktree where it was discovered_.
557+
558+
In the example above, the `modified-unit` will be discovered in a "to" temporary directory (e.g. `/tmp/.../terragrunt-worktree-HEAD.../modified-unit`), whereas the `removed-unit` would be discovered in the "from" temporary directory (e.g. `/tmp/.../terragrunt-worktree-main.../removed-unit`).
559+
560+
This is important to recognize, as it's how destroys will be possible despite the fact that the unit is no longer present in the current working directory. As a consequence, however, you may find that usage of absolute paths don't work how you expect, as you will be performing runs in the temporary directories created for the relevant worktrees.
481561

482-
<Aside type="note">
483-
Note that these commands _may_ have different logic for _discovery_, which is the process of surfacing the initial set of components to filter, but the filtering logic itself should be the same.
484562
</Aside>
485563

564+
## Usage with Commands
565+
566+
The following commands all support the `--filter` flag using the same filtering syntax (note the section below on [special interactions](#special-interactions)):
567+
568+
- [find](/docs/reference/cli/commands/find)
569+
- [list](/docs/reference/cli/commands/list)
570+
- [run](/docs/reference/cli/commands/run)
571+
- [hcl fmt](/docs/reference/cli/commands/hcl/fmt)
572+
- [hcl validate](/docs/reference/cli/commands/hcl/validate)
573+
574+
This flag is intended to be a flexible way to target specific infrastructure that allows you to dry-run infrastructure targeting using discovery commands (like `find` and `list`) before running a command that actually affects infrastructure (like `run`).
575+
486576
## Comparison with Queue Control Flags
487577

488-
The `--filter` flag provides a unified alternative to multiple queue control flags:
578+
The `--filter` flag provides a unified alternative to multiple queue control flags. These flags will be aliased to their equivalent filter expressions once the `filter-flag` experiment is stabilized:
489579

490580
| Legacy Flag | Filter Equivalent |
491581
|-------------|-------------------|
492-
| `--queue-include-dir` | `--filter './path/*'` |
493-
| `--queue-exclude-dir` | `--filter '!./path/*'` |
494-
| `--queue-exclude-external` | `--filter '!external=true'` |
495-
| `--queue-include-external` | `--filter 'external=true'` |
582+
| `--queue-include-dir=./path` | `--filter='./path'` |
583+
| `--queue-exclude-dir=./path` | `--filter='!./path'` |
584+
| `--queue-exclude-external` | `--filter='!external=true'` |
585+
| `--queue-include-external` | `--filter='external=true'` |
586+
| `--queue-include-units-reading=shared.hcl` | `--filter='reading=shared.hcl'` |
496587

497588
## Special Interactions
498589

0 commit comments

Comments
 (0)