Skip to content

Commit d1e9af7

Browse files
committed
docs: Documenting central Git store
1 parent 343dcdf commit d1e9af7

2 files changed

Lines changed: 27 additions & 5 deletions

File tree

docs/src/content/docs/03-features/07-caching/04-cas.mdx

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -102,15 +102,15 @@ Setting `update_source_with_cas = true` requires that the `cas` experiment is en
102102

103103
</Since>
104104

105-
When Terragrunt clones a repository while using the CAS, if the repository is not found in the CAS, Terragrunt will clone the repository from the original URL and store it in the CAS for future use.
105+
When Terragrunt clones a repository while using the CAS, if the repository is not found in the CAS, Terragrunt fetches into the central bare repository for that remote URL and stores the resulting blobs and trees in the CAS for future use. If the central store is unavailable, Terragrunt falls back to cloning the repository from the original URL into a temporary directory.
106106

107107
When generating a repository from the CAS, Terragrunt will hard link entries from the CAS to the new repository. This allows Terragrunt to deduplicate content across multiple repositories.
108108

109109
In the event that hard linking fails due to some operating system / host incompatibility with hard links, Terragrunt will fall back to performing copies of the content from the CAS.
110110

111111
## Storage
112112

113-
The CAS is stored in the `~/.cache/terragrunt/cas` directory. This directory can be safely deleted at any time, as Terragrunt will automatically regenerate the CAS as needed.
113+
The CAS is stored under the platform user cache directory. On Linux this is `~/.cache/terragrunt/cas` by default and honors `XDG_CACHE_HOME`, on macOS it resolves to `~/Library/Caches/terragrunt/cas`, and on Windows it resolves under `%LocalAppData%\terragrunt\cas`. This directory can be deleted to reclaim disk space when no Terragrunt processes are running against it; Terragrunt will regenerate the CAS on the next run. Avoid deleting it while a Terragrunt operation is in progress, since that can race with in-flight reads, writes, and locks in the store.
114114

115115
Avoid partial deletions of the CAS directory without care, as that might result in partially cloned repositories and unexpected behavior.
116116

@@ -179,10 +179,14 @@ The CAS store is organized into namespaced directories:
179179
- trees/ (synthetic trees created during CAS-backed stack generation)
180180
- de/
181181
- def456...xyz
182+
- git/ (one bare repository per remote URL, used for incremental fetches)
183+
- 1a2b3c4d5e6f7890/
184+
- repo/
185+
- lock
182186

183187
</FileTree>
184188

185-
The `blobs/` directory stores all file content, identified by hash. Blobs are purely content-addressed, so the same file content always maps to the same hash regardless of origin. The `trees/` directory stores Git-derived tree structures that describe the layout of files in a repository. The `synth/trees/` directory stores synthetic tree structures created during CAS-backed stack generation when `update_source_with_cas` is used. These synthetic trees use a deterministic hash based on the Git reference and path within the repository.
189+
The `blobs/` directory stores all file content, identified by hash. Blobs are purely content-addressed, so the same file content always maps to the same hash regardless of origin. The `trees/` directory stores Git-derived tree structures that describe the layout of files in a repository. The `synth/trees/` directory stores synthetic tree structures created during CAS-backed stack generation when `update_source_with_cas` is used. These synthetic trees use a deterministic hash based on the Git reference and path within the repository. The `git/` directory holds one bare Git repository per remote URL, keyed by a hash of the URL, so cache misses can fetch only the new objects instead of re-cloning the repository.
186190

187191
Each content object within a namespace is stored at `{hash[:2]}/{hash}`, where the first two characters create a partition directory to avoid degraded file system performance from large flat directories.
188192
</Since>
@@ -197,11 +201,13 @@ For cold clones, where the content is not already in the CAS:
197201

198202
1. Terragrunt resolves the Git reference (branch/tag) to a commit hash
199203
2. The tree related to the commit hash is not found in the CAS
200-
3. Terragrunt clones the repository to a temporary directory
201-
4. All blobs and trees required to reproduce the repository are extracted
204+
3. Terragrunt opens the bare repository for the remote URL under `cas/store/git/` (initializing it on first use), takes a per-URL lock, and fetches the requested ref. Subsequent misses against the same URL reuse the existing pack files and only transfer new objects.
205+
4. All blobs and trees required to reproduce the repository are extracted from the bare repository
202206
5. Content is stored in the CAS, partitioned by hash prefix
203207
6. The tree structure is read from the CAS and hard links are created to the target directory
204208

209+
If the central Git store cannot be used (for example, the lock cannot be acquired or the fetch fails), Terragrunt logs a warning and falls back to a clone in a temporary directory.
210+
205211
#### Warm Clones
206212

207213
For warm clones, where the content is already in the CAS:
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
---
2+
version: "v1.0.4"
3+
category: "experiments-updated"
4+
---
5+
6+
#### CAS keeps a central Git store for incremental fetches
7+
8+
CAS now keeps one bare Git repository per remote URL inside its store, under `~/.cache/terragrunt/cas/store/git/` on Linux by default. See [Storage](/features/caching/cas#storage) for where this lives on macOS and Windows. On a cache miss, Terragrunt fetches just the requested ref into that repository instead of running a fresh shallow clone into a temporary directory. Repeated misses against the same remote reuse the existing pack files, so fetching a second ref from the same repository transfers only the new objects.
9+
10+
Concurrent Terragrunt invocations are coordinated by a per-URL lock so pack-file writes do not interleave. If the central store cannot be used for any reason, Terragrunt logs a warning and falls back to the previous temporary-clone path so cloning still succeeds.
11+
12+
You can reclaim space at any time by deleting the `git/` subdirectory:
13+
14+
```bash
15+
rm -rf ~/.cache/terragrunt/cas/store/git
16+
```

0 commit comments

Comments
 (0)