Skip to content

Add configuration option to skip search paths - #107

Merged
django23 merged 23 commits into
AsimovMac:mainfrom
lunaluxie:main
Jul 30, 2026
Merged

Add configuration option to skip search paths#107
django23 merged 23 commits into
AsimovMac:mainfrom
lunaluxie:main

Conversation

@lunaluxie

@lunaluxie lunaluxie commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Add a [skip_paths] configuration option to allow the user to extend the skipped paths from the search.

The documentation and test-suite has been updated to reflect the changes. And I have checked for regressions. The PR has been rebased onto the latest commit on main.

I am not sure about the level of granularity wanted for the commits. Let me know if you need me to squash or reword the commits to better reflect the conventions of this project.

Benefits

  • The user can run Asimov in the home directory while excluding quarantined directories such as Photos, Documents, etc. This avoids permission errors, or having to give access to those directories reducing the attack surface.
  • Excluding large sub-directories which are irrelevant to Asimov can modestly speed up execution time.

Considerations

  • If code repositories are all located in one subfolder eg. ~/git, or ~/Developer, then it may be more appropriate to run Asimov only in that folder. This has been explained in the readme.
  • While Asimov internally uses skip_paths to refer to paths skipped during search, it may be more appropriate to call the user-facing configuration option skip_dirs for consistency with the existing fixed_dirs configuration option.

Test plan

  • make check passes

  • [?] Shellcheck clean (ran make lint which passes)

  • CHANGELOG.md updated (if user-facing)

  • Add test to verify behavior (skips directories given in [skip_paths] config behavior test)

  • Update readme/changelog to reflect changes.

@django23

Copy link
Copy Markdown
Collaborator

@lunaluxie thanks for this, and sorry for the wait — it's still wanted. 🙏

Heads-up that v0.10.0 changed the ground under this PR: there's a real config parser now (~/.config/asimov/config, with [sentinels] and [fixed_dirs] sections), so skip paths should slot in as another key rather than a separate mechanism:

[skip_paths]
extra = ~/Applications

If you're up for rebasing onto main in that shape, I'd be glad to review and land it. This would also close out #14, and is the natural counterpart to the just-shipped [scan] (#108).

Totally fine if you'd rather not — just say so and I'll pick it up. 🙏

django23 and others added 16 commits July 30, 2026 11:23
Add a [scan] section to ~/.config/asimov/config with one `extra =` line per
additional root; each is scanned on every run alongside the home directory.

- Resolve all roots into a single ASIMOV_SCAN_DIRS array; every consumer
  (Spotlight cache, path cache scoping, mdfind discovery, find traversal) loops
  over it instead of a lone ASIMOV_SCAN_DIR
- A positional CLI argument still overrides everything (scans only that path)
- Configured dirs that don't exist are warned about (honoring --quiet) and
  skipped, so an unmounted volume doesn't abort the run
- Nested/duplicate roots are pruned so no tree is traversed twice

Closes AsimovMac#108
- CHANGELOG: promote [scan] and Go-cache entries into [0.9.0], date 2026-07-26
- README: Install split into Homebrew (homebrew-core) + curl; document [scan];
  drop the retired django23 tap; note the pending homebrew-core 0.3.0→0.9.0 bump
- UPGRADING: point install commands at the stable curl one-liner
- CONTRIBUTING: Homebrew distribution is homebrew-core; mark tap targets legacy
- Makefile: verify-release falls back to `brew install asimov` (core), not the tap
Stable release carrying the v0.9.0-beta fixes plus the new [scan] multi-scan
feature. A new feature is a minor bump under semver, so the beta line's 0.9.0
becomes 0.10.0 for stable (there was no 0.9.0 stable).
- Makefile: remove bump-formula/ship-formula/TAP_DIR (tap archived); release
  target points at homebrew-core autobump; verify-release notes the ~3h lag
- README: homebrew note is now version-agnostic (formula auto-updates)
- CONTRIBUTING: release flow drops the tap steps; document that asimov is on
  Homebrew's autobump list (version PRs are automatic; only metadata needs a
  manual PR)
asimov starts with #!/usr/bin/env bash, so it runs under whichever bash comes
first on PATH. On macOS that is one of two very different things: /bin/bash,
still 3.2 because Apple froze it at the last GPLv2 release and what most users
get, or a 5.x build from Homebrew, which is what a development machine usually
has. They disagree on array and IFS semantics, so until now a change could pass
`make check` locally and CI and still be broken for users.

CI now runs os × bash (4 cells, all concurrent, fail-fast off so one failing
combination cannot hide the others). ShellCheck moves into its own job because it
is interpreter-independent and only needs to run once; it runs alongside the test
cells rather than four times over. A concurrency group cancels an in-flight run
when the same ref is pushed again.

Locally, `make test-system-bash` runs the suite under 3.2, and
`make test BASH_BIN=<path>` pins any interpreter. Both go through the new
scripts/test.sh, which sets ASIMOV_TEST_BASH; the Bats helper then launches
asimov with that interpreter explicitly, so Bats itself keeps running under its
own bash rather than being dragged down to 3.2 by a PATH shim.

`make test BATS_JOBS=N` runs tests concurrently (needs GNU parallel). Each test
already gets its own temp HOME in setup(), so there is no shared state.
Covers the three most common misconceptions: Asimov sets Time Machine
exclusions only, it does not hide directories from Spotlight, and it does
not shrink backups that already exist. Adds a short verification recipe
for the "my node_modules is still in Time Machine" case.

Refs AsimovMac#90, AsimovMac#45
Reports Time Machine exclusions whose directory no longer exists and
compacts Asimov's own path cache.

The premise of AsimovMac#38 turned out not to hold for Asimov's own work: every
version in git history calls `tmutil addexclusion PATH`, which stores the
exclusion as an extended attribute on the directory, so deleting the
directory deletes the exclusion with it. Nothing is left behind.

Staleness lives in the other mechanism — `tmutil addexclusion -p`, which
records the path in Time Machine's system preferences and survives the
directory forever. Asimov has never written that list, but other tools and
manual commands do, and nothing surfaces the leftovers.

So prune reports rather than removes: the entries are system-wide and not
ours to delete. It prints the `sudo tmutil removeexclusion -p` command
instead. The only file it writes is Asimov's own cache.

Closes AsimovMac#38
The cache under ~/.cache/asimov is an optimisation, but nothing treated it
as optional. A bare `cat` on an unreadable state file failed under
`set -Eeu -o pipefail` and aborted the run immediately, printing nothing
but "Permission denied" — no context, no fix, exit 1.

The cause was Asimov itself. ensure_cache_dir chowned the cache directory
to the console user when running as root, but the state files were created
*after* that chown, so every `sudo asimov` left root-owned files behind and
broke the next run as the user.

Guard every cache read and write behind cache_readable/cache_writable, warn
once naming the reset command, and carry on without the cache. Create the
state files before the chown so appends — which never change an existing
file's owner — keep them owned by the user who has to read them next.

Closes AsimovMac#122
Checks the install rather than the projects: which asimov the shell
actually runs, whether a schedule is installed and loaded, whether the
cache is readable and writable, whether the config parses, and whether
tmutil can read exclusions at all. Exits 1 if it finds anything.

Two rules keep it safe to run part-way through a migration. It never
writes — fixes are printed, not applied. And it never executes another
asimov binary it finds: v0.3.0 parses no arguments at all, so running it
to ask its version would start a real scan, so versions are read out of
the file instead.

The test helper now strips any real asimov install from PATH, since doctor
inspects PATH and would otherwise report a different result per machine.

Refs AsimovMac#122
v0.3.0 is the version most people have, and it parses no arguments at all:
`asimov --version` and `asimov doctor` are both ignored and it goes
straight to scanning. So the upgrade notes lead with a version check that
reads the file instead of running it, and put `asimov doctor` last, after
the new binary is in place.

Also documents the three leftovers that outlive a v0.3.0 install — the
LaunchAgent, the old cellar, and a root-owned cache — and that Asimov is a
one-shot scan, not a daemon, so an empty `ps aux | grep asimov` after
`brew services start` is expected.

Refs AsimovMac#122
Every other script under tests/bin/ is linted; the launchctl mock added
alongside `asimov doctor` was missed.

Co-authored-by: django23 <827397+django23@users.noreply.github.com>
* test(doctor): derive the expected version from the script

The version assertion hardcoded 0.10.0, so it failed the moment the
release bump landed. Read it from `asimov --version` instead.

* docs: release 0.11.0

---------

Co-authored-by: django23 <827397+django23@users.noreply.github.com>
@lunaluxie

Copy link
Copy Markdown
Contributor Author

@django23 Thank you for taking the time to review my pull request.

I have attempted to rebase onto upstream/main, and I have manually verified that my changes still work, and I have rerun the tests with make check.

However, I am not quite sure what you mean by reworking the config parser. It appears that the main branch still uses the switch case mechanism which skip_paths was already slotted into.

case "${section}:${key}" in
                fixed_dirs:enabled)
                    ASIMOV_CONFIG_FIXED_DIRS_ENABLED="$value"
                    ;;
                fixed_dirs:extra)
                    value="${value/#\~/$HOME}"
                    ASIMOV_CONFIG_EXTRA_FIXED_DIRS+=("$value")
                    ;;
                scan:extra)
                    value="${value/#\~/$HOME}"
                    ASIMOV_CONFIG_SCAN_DIRS+=("$value")
                    ;;
                sentinels:extra)
                    ASIMOV_CONFIG_EXTRA_SENTINELS+=("$value")
                    ;;
                sentinels:disabled)
                    ASIMOV_CONFIG_DISABLED_SENTINELS+=("$value")
                    ;;
                skip_paths:extra)
                    value="${value/#\~/$HOME}"
                    ASIMOV_CONFIG_EXTRA_SKIP_PATHS+=("$value")
                    ;;
            esac

Finally, I am a bit worried that I have made a mistake in rebasing. I can see that my pull request now includes all the new changes from main as well which I doubt is the proper way to do it. But I haven't used git before in a collaborative environment, so I am not sure what I have done wrong.

…or about it

Follow-up to the [skip_paths] work: append the configured paths to
ASIMOV_SKIP_PATHS at definition time instead of merging them into a local
array in each consumer, so the find expression and the Spotlight pass stay
in step, add the section to the doctor config validator (it reported
[skip_paths] as an unknown section), and cover both in tests.

The behaviour test passed an absolute path to create_project, which prefixes
$HOME - so the project was built outside the skipped directory and the test
passed for the wrong reason. Fixed, plus a control case that fails without
the config, a multi-entry case, and one proving [fixed_dirs] still wins
inside a skipped path.
@django23

django23 commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

@lunaluxie you rebased it correctly - nothing to fix. The commit list shows main's commits because of the merge commit, but the actual diff (gh pr diff 107) is still just your 4 files. That's the thing that gets reviewed and merged, so you're good. 🙏

And you're right about the parser: [skip_paths] slots straight into the existing case "${section}:${key}", which is exactly what you did. My earlier comment described work you'd already done - my mistake for not re-reading the branch first.

Two things I've pushed on top of yours:

  • asimov doctor didn't know about the new section. It validates the config against an allowlist and was reporting unknown section [skip_paths], exit 1.
  • The behaviour test passed for the wrong reason. create_project prefixes $HOME, so an absolute first argument built the project at $HOME/$HOME/.skipme/... - outside the skipped path. Now it uses a relative base, with a control case that fails when the config is absent.

I also merged the configured paths into ASIMOV_SKIP_PATHS at the point it's defined rather than in each consumer, so find and the Spotlight pass can't drift apart:

readonly ASIMOV_SKIP_PATHS=(
    "${ASIMOV_ROOT}/.Trash"
    "${ASIMOV_ROOT}/Library"
    ${ASIMOV_CONFIG_EXTRA_SKIP_PATHS[@]+"${ASIMOV_CONFIG_EXTRA_SKIP_PATHS[@]}"}
)

make check is green (225 tests + shellcheck). This closes #14 too.

Thanks for sticking with it. 🎉
If you agree with the changes too, I'll merge.

…yout

The monolithic `asimov` is gone as of AsimovMac#126, so this resolves the
modify/delete conflict by re-landing the feature where it now belongs:

- lib/asimov/config.sh parses `skip_paths:extra`
- lib/asimov/data.sh appends those paths to ASIMOV_SKIP_PATHS in
  load_skip_paths, after the data/skip-paths.tsv rows. load_config runs
  before load_data, so the array is complete by the time scan.sh and
  discover.sh read it, and neither needs to know about the config
- lib/asimov/doctor.sh accepts the section and its one key

Tests and docs carried over unchanged. 238 tests pass under bash 5 and
under the system bash 3.2.
@django23
django23 merged commit 81b5027 into AsimovMac:main Jul 30, 2026
5 checks passed
@django23 django23 mentioned this pull request Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

2 participants