Skip to content

Commit b7a9fd6

Browse files
committed
fix: skip read-only dirs and stop tmutil's POSIXError leaking into output
Go's module cache is deliberately 0555. A Time Machine exclusion is stored as an xattr on the item itself, so a read-only directory can never take one — tmutil fails with "Error (-20)" / EINVAL(22) after burning ~11s per call. The failures were long attributed to the '@' in Go module version paths. That was wrong: a writable path containing '@' excludes fine; permissions were always the cause. Comments corrected and both cases now have regression tests. - Pre-check writability and skip with an accurate message, before the ~11s call - Silence tmutil's stdout too: it prints its POSIXError dump to stdout, not stderr, so the existing 2>/dev/null never suppressed it - Add ~/go/pkg/mod to ASIMOV_FIXED_DIRS — excludes the cache in one call and lets the existing Layer 2 filter drop every vendor/ dir beneath it - Mock tmutil now mirrors the real stream split (dump on stdout, error on stderr) so the suite can actually catch this class of leak - README: split Install into Stable / Beta / Quick start; refresh Credits for the handover; document the Go cache and custom-GOPATH config
1 parent 824e0dc commit b7a9fd6

5 files changed

Lines changed: 137 additions & 18 deletions

File tree

CHANGELOG.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,16 @@ This project adheres to [Semantic Versioning](http://semver.org/).
88

99
### Added
1010

11+
- Go's module cache (`~/go/pkg/mod`) is now one of the built-in global caches (opt-in via `[fixed_dirs] enabled = true`). Excluding the cache root covers every dependency inside it in a single `tmutil` call, instead of attempting each `vendor/` directory individually (~11s apiece, and doomed — see below). Using a custom `GOPATH`? Add it with `[fixed_dirs] extra`
12+
1113
### Changed
1214

1315
### Fixed
1416

17+
- Read-only directories are now detected and skipped up front with a clear `read-only, cannot be excluded` message, instead of burning ~11s on a `tmutil addexclusion` call that can never succeed. A Time Machine exclusion is stored as an extended attribute **on the item itself**, so a `0555` directory can never take one — Go's module cache is deliberately read-only, which is why its `vendor/` directories always failed
18+
- `tmutil`'s `POSIXError(_nsError: …Code=22 "Invalid argument")` dump no longer leaks into Asimov's output. `tmutil` prints it to **stdout** (not stderr), so the existing `2>/dev/null` never suppressed it; both streams are now silenced and Asimov prints its own warning instead
19+
- Corrected a long-standing misdiagnosis in the code comments: these failures were attributed to the `@` in Go module version paths. `@` is irrelevant — a writable path containing `@` excludes fine. Read-only permissions were always the cause (regression tests now cover both)
20+
1521
### Removed
1622

1723
## [0.9.0] — 2026-07-09

README.md

Lines changed: 31 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -15,14 +15,36 @@ Asimov scans your home directory, finds dependency folders next to their config
1515

1616
## Install
1717

18-
> **Beta:** `v0.9.0` is in pre-release testing. Install it with the curl command below.
18+
### Stable — v0.8.0
19+
20+
```sh
21+
brew install django23/tap/asimov
22+
brew services start django23/tap/asimov
23+
```
24+
25+
> `brew install asimov` from homebrew-core is still **v0.3.0** and unmaintained. Use the tap above.
26+
27+
### Beta — v0.9.0-beta.1
28+
29+
Currently in testing, and where the newest ecosystems and the ~1–2s cached runs live:
1930

2031
```sh
2132
curl -fsSL https://raw.githubusercontent.com/AsimovMac/asimov/v0.9.0-beta.1/scripts/install-remote.sh \
2233
| ASIMOV_REF=v0.9.0-beta.1 bash
2334
```
2435

25-
That's it — Asimov installs to `~/.local/bin` and runs at midday, every day. See [other install methods](#other-install-methods) for Homebrew and source installs.
36+
Installs to `~/.local/bin`. Feedback and bug reports very welcome — [open an issue](https://github.com/AsimovMac/asimov/issues).
37+
38+
### Quick start
39+
40+
```sh
41+
asimov --dry-run # preview what would be excluded — changes nothing
42+
asimov # apply
43+
```
44+
45+
That's it. Asimov then runs itself once a day, every day. Add `--stats` to either command to see sizes and how much space you're saving.
46+
47+
See [other install methods](#other-install-methods) to build from source.
2648

2749
## What you'll see
2850

@@ -79,7 +101,7 @@ This means Asimov never touches a folder that just happens to share a common nam
79101
| **R** | `renv` |
80102
| **DevOps / IaC** | `.terraform`, `.terragrunt-cache`, `.vagrant`, `.direnv`, `cdk.out` |
81103
| **Game dev** | `.godot` |
82-
| **Global caches** (opt-in) | `~/.cache`, `~/.gradle/caches`, `~/.m2/repository`, `~/.npm/_cacache`, `~/.nuget/packages`, `~/.kube/cache` |
104+
| **Global caches** (opt-in) | `~/.cache`, `~/.gradle/caches`, `~/.m2/repository`, `~/.npm/_cacache`, `~/.nuget/packages`, `~/.kube/cache`, `~/go/pkg/mod` |
83105

84106
**Don't see your tool?** You can teach Asimov your own directory + sentinel pairs in a couple of lines — no need to wait for a release. See [Add your own patterns](#add-your-own-patterns).
85107

@@ -146,17 +168,11 @@ Global tool caches in your home directory (`~/.cache`, `~/.gradle/caches`, …)
146168
[fixed_dirs]
147169
enabled = true # exclude the built-in global caches
148170
extra = ~/my-build-cache # plus any paths you name (always excluded when they exist)
171+
extra = ~/golang/pkg/mod # e.g. a Go module cache under a custom GOPATH
149172
```
150173

151174
## Other install methods
152175

153-
**Homebrew** — the tap currently tracks the last stable (`0.8.0`), not the `0.9.0` beta:
154-
155-
```sh
156-
brew install django23/tap/asimov
157-
brew services start django23/tap/asimov
158-
```
159-
160176
**From source:**
161177

162178
```sh
@@ -179,7 +195,11 @@ See [UPGRADING.md](UPGRADING.md) for migrating from v0.4.x or the original `stev
179195

180196
## Credits
181197

182-
Asimov was created by [Steve Grunwell](https://github.com/stevegrunwell) and maintained at [`stevegrunwell/asimov`](https://github.com/stevegrunwell/asimov) through v0.3.0. Versions since `0.4.0` add performance improvements, config-file support, and expanded ecosystem coverage.
198+
Asimov was created by **[Steve Grunwell](https://github.com/stevegrunwell)**, who built and maintained it at `stevegrunwell/asimov` from 2018 through v0.3.0 — the version most people know it by. Thank you, Steve, for the years of work and for [handing it on](https://github.com/AsimovMac/asimov/issues/99) so carefully.
199+
200+
The project now lives at [`AsimovMac/asimov`](https://github.com/AsimovMac/asimov) (the original repo, transferred — the old URL still redirects here), maintained by [@django23](https://github.com/django23). Releases from `v0.4.0` onward add expanded ecosystem coverage, a config file, and the caching that took typical runs from ~75s to ~1–2s.
201+
202+
Built on the contributions of everyone who filed issues and PRs over the years. Asimov is, and stays, MIT-licensed and community-driven.
183203

184204
## Contributing
185205

asimov

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -175,7 +175,7 @@ mdfind -onlyin "${ASIMOV_SCAN_DIR}" "com_apple_backup_excludeItem = 'com.apple.b
175175
| sort -u > "$ASIMOV_EXCLUDED_CACHE" || true
176176
spotlight_count="$(wc -l < "$ASIMOV_EXCLUDED_CACHE" | tr -d ' ')"
177177
# Merge persistent excluded + failed state into the Spotlight cache.
178-
# Failed paths (e.g. Go modules with @ chars) are included so the bulk filter
178+
# Failed paths (e.g. read-only dirs inside Go's module cache) are included so the bulk filter
179179
# skips them instantly instead of wasting time retrying tmutil.
180180
# Skipped under --no-read-cache (--full-scan / --no-cache): every path is then
181181
# re-verified against the tmutil isexcluded ground truth instead.
@@ -620,6 +620,7 @@ readonly ASIMOV_FIXED_DIRS=(
620620
"${ASIMOV_ROOT}/.nuget/packages" # NuGet global packages
621621
"${ASIMOV_ROOT}/.kube/cache" # Kubernetes API cache
622622
"${ASIMOV_ROOT}/.kube/http-cache" # Kubernetes HTTP cache
623+
"${ASIMOV_ROOT}/go/pkg/mod" # Go module cache (0555 inside; exclude as a whole)
623624
)
624625

625626
# Record an excluded path: log for counting, compute size only with --stats.
@@ -648,8 +649,8 @@ record_excluded_path() {
648649
# Performance notes (learned the hard way):
649650
# - tmutil addexclusion takes ~11s per call (IPC with TM daemon). This dominates runtime.
650651
# - Batching (tmutil addexclusion path1 path2 ...) provides NO speed benefit — tmutil
651-
# processes paths sequentially at ~11s each. Worse: one bad path (e.g. Go modules with
652-
# @ characters) fails the entire batch, wasting all accumulated time before fallback.
652+
# processes paths sequentially at ~11s each. Worse: one bad path (e.g. a read-only
653+
# dir in Go's module cache) fails the entire batch, wasting all accumulated time.
653654
# - Spotlight (mdfind) can be stale: recently excluded paths may not appear in the index.
654655
# Without a guard, this causes re-excluding already-done paths (~11s each, wasted).
655656
# - Three-layer defense against wasted tmutil calls:
@@ -748,10 +749,20 @@ exclude_paths_from_stdin() {
748749
continue
749750
fi
750751
local path_start=$SECONDS
751-
if ! tmutil addexclusion "${path}" 2>/dev/null; then
752+
# Time Machine stores an exclusion as an xattr on the item itself, so a
753+
# read-only directory can never be excluded — tmutil fails with
754+
# "Error (-20)" / EINVAL. The common case is Go's module cache, which is
755+
# deliberately 0555. Check first: addexclusion costs ~11s even when it fails.
756+
if [[ ! -w "$path" ]]; then
757+
echo "! ${path}: read-only, cannot be excluded — skipping." >&2
758+
[[ -z "$ASIMOV_NO_WRITE_CACHE" ]] && printf '%s\n' "$path" >> "$ASIMOV_FAILED_STATE"
759+
continue
760+
fi
761+
# tmutil prints its POSIXError dump to stdout, not stderr — silence both.
762+
if ! tmutil addexclusion "${path}" >/dev/null 2>&1; then
752763
echo "! ${path}: failed to exclude (tmutil error), skipping." >&2
753764
# Persist failure so we skip this path on subsequent runs.
754-
# Common cause: Go module paths with @ characters. Use --no-read-cache to retry.
765+
# Use --no-read-cache to retry.
755766
[[ -z "$ASIMOV_NO_WRITE_CACHE" ]] && printf '%s\n' "$path" >> "$ASIMOV_FAILED_STATE"
756767
continue
757768
fi

tests/behavior.bats

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -233,6 +233,28 @@ enabled = true"
233233
[[ "$(count_exclusions)" -eq 3 ]]
234234
}
235235

236+
@test "excludes the Go module cache when fixed dirs are enabled" {
237+
mkdir -p "${HOME}/go/pkg/mod"
238+
write_config "[fixed_dirs]
239+
enabled = true"
240+
run_asimov
241+
assert_excluded "${HOME}/go/pkg/mod"
242+
}
243+
244+
@test "excludes the Go module cache as a whole, not each vendor dir inside it" {
245+
# Real-world shape: the module cache holds many read-only vendor/ dirs behind
246+
# @-versioned paths. Excluding the cache root covers all of them in one call;
247+
# excluding each individually would be ~11s apiece and fail on the 0555 perms.
248+
create_project "go/pkg/mod/github.com/foo/bar@v1.2.3" "go.mod" "vendor"
249+
write_config "[fixed_dirs]
250+
enabled = true"
251+
252+
run_asimov
253+
254+
assert_excluded "${HOME}/go/pkg/mod"
255+
refute_excluded "${HOME}/go/pkg/mod/github.com/foo/bar@v1.2.3/vendor"
256+
}
257+
236258
@test "does not re-exclude already excluded fixed directory" {
237259
mkdir -p "${HOME}/.cache"
238260
write_config "[fixed_dirs]
@@ -407,6 +429,62 @@ extra = .custom-deps *.x'; touch ${pwned}; '"
407429
[[ "$(count_exclusions)" -eq 0 ]]
408430
}
409431

432+
@test "does not leak tmutil's POSIXError dump into output" {
433+
create_project "Code/Bad-Project" "package.json" "node_modules"
434+
435+
ASIMOV_TEST_TMUTIL_FAIL_PATHS="${TEST_TEMP_DIR}/.tmutil_fail_paths"
436+
export ASIMOV_TEST_TMUTIL_FAIL_PATHS
437+
echo "${HOME}/Code/Bad-Project/node_modules" > "$ASIMOV_TEST_TMUTIL_FAIL_PATHS"
438+
439+
run_asimov
440+
441+
[[ "$status" -eq 0 ]]
442+
# tmutil dumps POSIXError(...) to stdout — not stderr — so a bare 2>/dev/null
443+
# lets it leak. Asimov must swallow it and print its own warning instead.
444+
[[ "$output" != *"POSIXError"* ]]
445+
[[ "$output" == *"failed to exclude"* ]]
446+
}
447+
448+
# =============================================================================
449+
# Read-only directories (e.g. Go's module cache, which is 0555)
450+
# =============================================================================
451+
452+
@test "skips read-only directory without attempting tmutil" {
453+
create_project "Code/Go-Project" "go.mod" "vendor"
454+
chmod 555 "${HOME}/Code/Go-Project/vendor"
455+
456+
run_asimov
457+
local status_copy="$status" output_copy="$output"
458+
chmod 755 "${HOME}/Code/Go-Project/vendor"
459+
460+
[[ "$status_copy" -eq 0 ]]
461+
refute_excluded "${HOME}/Code/Go-Project/vendor"
462+
# The read-only pre-check should fire — NOT the tmutil-failure path. A Time
463+
# Machine exclusion is an xattr on the item, so a 0555 dir can never take one.
464+
[[ "$output_copy" == *"read-only"* ]]
465+
[[ "$output_copy" != *"tmutil error"* ]]
466+
}
467+
468+
@test "read-only directory is recorded in failed state" {
469+
create_project "Code/Go-Project" "go.mod" "vendor"
470+
chmod 555 "${HOME}/Code/Go-Project/vendor"
471+
472+
run_asimov
473+
chmod 755 "${HOME}/Code/Go-Project/vendor"
474+
475+
assert_failed "${HOME}/Code/Go-Project/vendor"
476+
}
477+
478+
@test "writable directory containing @ is excluded (@ is not the blocker)" {
479+
# Regression guard: the failure on Go module paths was long attributed to the
480+
# '@' in the version syntax. It is not — read-only permissions are the cause.
481+
create_project "Code/cheat@v0.0.0-20211009161301" "go.mod" "vendor"
482+
483+
run_asimov
484+
485+
assert_excluded "${HOME}/Code/cheat@v0.0.0-20211009161301/vendor"
486+
}
487+
410488
# =============================================================================
411489
# --help, --version, unknown option
412490
# =============================================================================

tests/bin/tmutil

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,8 +45,12 @@ case "$subcommand" in
4545
# Simulate failure for paths listed in ASIMOV_TEST_TMUTIL_FAIL_PATHS
4646
if [[ -n "${ASIMOV_TEST_TMUTIL_FAIL_PATHS:-}" && -f "${ASIMOV_TEST_TMUTIL_FAIL_PATHS}" ]]; then
4747
if grep -Fxq "$path" "$ASIMOV_TEST_TMUTIL_FAIL_PATHS" 2>/dev/null; then
48-
echo "Error (-50) while attempting to change exclusion setting." >&2
49-
exit 1
48+
# Mirror real tmutil: the POSIXError dump goes to STDOUT (not stderr),
49+
# while the "Error (-NN)" line goes to stderr. Keeping both faithful is
50+
# what lets tests catch the dump leaking into Asimov's output.
51+
echo 'POSIXError(_nsError: Error Domain=NSPOSIXErrorDomain Code=22 "Invalid argument")'
52+
echo "${path}: Error (-50) while attempting to change exclusion setting." >&2
53+
exit 22
5054
fi
5155
fi
5256
# Idempotent: don't add if already present (matches real tmutil)

0 commit comments

Comments
 (0)