Ansible Autoconfig

Ansible Autoconfig

This repository is based on the code that Jay Lacroix (LearnLinuxTV) worked on from his Ansible Desktop tutorial on Youtube.

Additionnal inspiration could be taken from Jeef Geerling's work on Ansible

It was improved with the Ansible-pull tutorial and then coding was improved with ChatGPT's Codex (via opencode and pi.dev).

Usage

objective is to have something that can be run easily on a new machine or not (idempotent) with:

sudo ansible-pull -d /opt/ansible-pull -U https://github.com/djspatule/ansible-autoconfig.git -i hosts local.yml --vault-password-file ~/secret.txt

The -d /opt/ansible-pull matters: run as sudo, ansible-pull otherwise clones into /root/.ansible/pull/… (mode 700), and the dotfiles stow step — which runs as your normal user — cannot read the repo there and fails with a permission error. Always check out into a shared, readable path. (The managed auto-pull timer already does this.)

in test mode, to avoid having to "git push" each modification in order to test it locally on a development machine where the repo is present (because i'm currently working on it for instance), use:

sudo ansible-pull -U "~/Documents/ansible-autoconfig/" --check

or

sudo ansible-playbook -i "localhost," -c local local.yml --check

remember that ansible-pull is just a wrapper for ansible-playbook....so both cmds are kinda similar ... and the check option allows to test on a system without implementing anything ('dry run').

ToDo

Guidelines for coding agents

Objectives

Use ansible-pull (sudo ansible-pull -U https://github.com/djspatule/ansible-autoconfig.git --vault-password-file ~/secret.txt -C main) to automate the set-up of any of my family’s computer if they ever have to be reinstalled from scratch (on the same or different hardware).
The set-up must be idem-potent so that it can be set to run automatically once a day on each of these machine (via a crontab job or else)
I need to be able to “test” on a virtual machine.

My dotfiles are currently specific to Omarchy and managed with stow (https://github.com/djspatule/omarchy-dotfiles)... I want to transition from the dotfiles specific repo to this repo and have stow managed by ansible as part of the setup. Maintenance of the files need to be feasible and easy.

Use this for inspiration of structuration, etc. https://github.com/LearnLinuxTV/personal_ansible_desktop_configs/tree/main
I’m writing this code also as an opportunity to learn about ansible, GNU/linux, neovim, opencode and other coding agents (and vibecoding in general), tmux, git, networking, etc. Thus explain all the code, design choices, structure, etc in that perspective...
I would ideally like to lay here the foundations of an auto-config ansible setup that will be still valid and usable in 20 years….(comment your code profusely in that regard)
The short term priority is to work on the server but the rest will be addressed as well (kids computer, adult's workstations, etc.).

Roles

Server (Serverannah) destined to become a homelab that runs Ubuntu server 25.10 (headless Optiplex 3070, accessed via SSH) : It is running both natively (on bare metal) an Nginx server to host 3 websites (including a complex wordpress) and more to come. It's also DNS Filtering with pi-hole (installed on bare metal). last, it's running multiple docker services such as frigate for cameras, N8N, Odoo, bentopdf, Plex, Timeshift (to secure backups on 1 master and 3 different external disks copies), Nextcloud (to manage my data and make it accessible to the whole family) and the like. For now, those functions are managed on a raspi 4B (192.168.1.100) that should be retired soon. You don’t have to keep on serverannah things as they were on the raspi. It’s a good opportunity to “improve”.
Kids are running Linux Mint (xfce but switch to cinnamon is envisaged) on a very old macbook. Their goal is gaming and learning. Parental control has to be strict
“Mac” is my wife’s computer (not much to set-up there, she mostly works in a browser + GDrive and Spotify)
Laptop is on Omarchy (hyprland) but could be switched to KDE or Gnome on Arch or even be switched to an Debian/Ubuntu distro

Extra instructions

ask whenever in doubt and ask for feedback (i can run commands for you, etc.)
verify your work as much as possible
remember as much as you can of these instructions and the other instructions to come
Create a “guidelines.md” inside the directory to describe everything the project is about, how it works, terminal commands, etc. Templates on github….

Do not do

do not change code without explaining
do not change multiples files at once when not strictly connected to the same taks/objective at hand

Agent written readme

This repository is the family ansible-pull source of truth.

The active goal is simple:

one main entry point: local.yml
one common role: roles/base
one server role: roles/server
code that stays readable by a non-expert
host-specific state stored in obvious places
configuration changed in git first, not directly on the machine

Design Rules

Prefer small Ansible modules over clever shell logic.
Keep one source of truth per concern.
Do not encode migration hacks for old machines unless they are still truly needed.
If a value is host-specific, keep it in host_vars/serverannah.
If a file is the real source of truth, store it in files/ and copy it.

Active Layout

local.yml: main playbook
hosts: active inventory
group_vars/all: tiny shared defaults only
host_vars/serverannah: server-specific switches, domains, URLs, and hardware paths
roles/base: packages, Starship shell init, dotfiles via Stow, ansible-pull automation
roles/server: Docker services, reverse proxy, Pi-hole, Nextcloud, Frigate, fail2ban, storage bootstrap
roles/workstation: first CLI/TUI and dotfiles foundation for Arch/Omarchy workstation testing
files/dotfiles/: Stow packages actually managed by this repo
files/serverannah/etc/fstab: source of truth for serverannah storage mounts
files/pihole/policy/: Pi-hole adlists and allowlist imported by Ansible

Current Entry Points

Run locally against the active inventory:

sudo ansible-playbook -i hosts --limit serverannah local.yml

Run with ansible-pull on the target machine:

sudo ansible-pull -U https://github.com/djspatule/ansible-autoconfig.git -C main -d /opt/ansible-pull local.yml

When testing on a VM or before public DNS/port forwarding is ready, do not ask Caddy to obtain public HTTPS certificates yet. Keep the real serverannah vars as the production target, and override only the staging network edge explicitly:

sudo ansible-pull -U https://github.com/djspatule/ansible-autoconfig.git -C main -d /opt/ansible-pull local.yml \
  -e 'server_reverse_proxy_auto_https=false server_reverse_proxy_published_ports=["8081:80"] server_reverse_proxy_sites=[{"hostname":"homepage.localtest.me","upstream":"homepage:3000"},{"hostname":"bentopdf.localtest.me","upstream":"bentopdf:8080"},{"hostname":"game-timer.localtest.me","upstream":"game-timer:80"},{"hostname":"pihole.localtest.me","upstream":"pihole:80"}]'

This keeps VM testing local and avoids confusing Let's Encrypt failures while the server is not publicly reachable.

Bootstrap a fresh server that does not have Ansible yet:

sudo ./scripts/bootstrap-server.sh

Dry-run a local checkout when testing:

sudo ansible-playbook -i hosts --limit serverannah local.yml --check

Secrets And Vault

This is a public repo, so secrets are handled at two levels:

Generated on the host, never committed. Service passwords (database passwords, Nextcloud admin password, Dinnizer secret key base, etc.) are generated on the target machine into /etc/ansible/secrets/ on first run and reused afterwards. They never enter git at all. This is the default and covers most services today.
Committed but encrypted with ansible-vault. For files that genuinely have to live in the repo (for example SSH config or private keys, API tokens in dotfiles), encrypt them with ansible-vault before committing. The encrypted blob is safe in a public repo; only the password that unlocks it is secret.

The vault password itself is never committed. Supply it one of two ways:

# Per command:
sudo ansible-playbook -i hosts local.yml --vault-password-file ~/.ansible-vault-pass

# Or for every run: uncomment vault_password_file in ansible.cfg.

The same --vault-password-file flows through scripts/bootstrap-server.sh (AUTOCONFIG_VAULT_PASSWORD_FILE) and the managed autoconfig-pull timer (autoconfig_vault_password_file), so a scheduled pull can decrypt vaulted files unattended. Password-file names like ~/.ansible-vault-pass, secret.txt and *.vault-pass are already in .gitignore.

Common commands:

ansible-vault encrypt files/dotfiles/ssh/.ssh/config   # encrypt in place
ansible-vault edit    files/dotfiles/ssh/.ssh/config   # edit encrypted
ansible-vault view    files/dotfiles/ssh/.ssh/config   # read without decrypting to disk

Keep vault_password_file commented in ansible.cfg until at least one encrypted file exists, otherwise runs fail looking for a password they do not need.

Linting

Linting is local-only. There is no GitHub Action and no cloud cost: everything runs on your machine.

scripts/lint.sh runs yamllint + ansible-lint. On first run it builds a throwaway virtualenv (.lint-venv/, gitignored) so nothing is installed system-wide. Run it any time:
```
./scripts/lint.sh
```
A pre-commit hook is an optional convenience: a check that git runs automatically every time you git commit, refusing the commit if linting fails (so mistakes never reach history). It is configured in .pre-commit-config.yaml and is also fully local. One-time setup:
```
pipx install pre-commit   # or: pip install --user pre-commit
pre-commit install        # installs the hook into .git/hooks
pre-commit run --all-files  # optional: lint everything now
```

Config lives in .yamllint (lenient: long lines warn, real booleans only) and .ansible-lint (basic profile; the role-prefix naming rule is deliberately skipped because this repo names variables by concern, e.g. nextcloud_, pihole_, not server_nextcloud_). The remaining findings are cosmetic (long lines, a few unnamed import_tasks) and can be cleaned up over time.

Testing On A VM

The workstation role is proven on a disposable Arch/Omarchy VM (savannarchome) before touching a real laptop. The VM configures itself with ansible-pull, exactly like a real new machine would, so no host-to-guest SSH or port-forwarding is needed.

GNOME Boxes uses QEMU user-mode networking (the guest gets 10.0.2.15, which the host cannot reach), so the self-pull model is not just convenient, it is the only simple option. Three lines inside the VM:

# 1. ansible-pull targets the inventory host whose name matches the machine's
#    hostname, so the VM must call itself savannarchome.
sudo hostnamectl set-hostname savannarchome

# 2. The vault password (same one used on the laptop) so config.local can decrypt.
printf '%s' 'YOUR_VAULT_PASSWORD' > ~/secret.txt && chmod 600 ~/secret.txt

# 3. Fetch and run the bootstrap: installs git+ansible, clones, installs
#    collections, runs ansible-pull for base + workstation.
curl -fsSL https://raw.githubusercontent.com/djspatule/ansible-autoconfig/main/scripts/bootstrap-server.sh \
  | sudo env AUTOCONFIG_VAULT_PASSWORD_FILE=/home/lion/secret.txt sh

A healthy first run ends with failed=0 and shows, among others, the Deploy vault-encrypted dotfiles (decrypted into place) task as changed (a wrong vault password would fail there). Reset or snapshot the VM freely between runs; the playbook is idempotent.

Live access from the host (for fast iteration): the self-pull above is enough to validate a change, but to drive --check runs against the VM from the host (or just poke around), a reverse SSH tunnel avoids fighting GNOME Boxes' NAT. GNOME Boxes user-mode networking reaches the host at 10.0.2.2, so run one command in the guest to dial out and forward host port 2222 back to the guest's sshd, then one on the host to install your key through it:

# In the GUEST: hold open a reverse tunnel (host:2222 -> guest:22).
ssh -fN -R 2222:127.0.0.1:22 lion@10.0.2.2

# On the HOST: authorize your key over that tunnel.
ssh-copy-id -p 2222 lion@127.0.0.1

After that, ssh -p 2222 lion@127.0.0.1 works, and because host_vars/savannarchome already points at 127.0.0.1:2222, so does ansible -i hosts savannarchome -m ping. The tunnel lasts until the VM reboots or that ssh process is killed; wrap it in autossh + a systemd user unit if you want it to persist.

Clipboard with GNOME Boxes (Hyprland guest): install spice-vdagent and wl-clipboard, enable spice-vdagentd, and add exec-once = spice-vdagent (without -x) to the Hyprland config so the client starts inside the Wayland session. Sync is then automatic (normal copy/paste, no special shortcut). If the client comes up in X11 mode (spice-vdagent -x), it cannot read the Wayland clipboard; pkill -x spice-vdagent then re-run spice-vdagent from a terminal in the live session.

Why `requirements.yml` Stays

requirements.yml is not just documentation.

It is the executable dependency manifest used by the bootstrap script and the managed autoconfig-pull helper to install required collections such as:

community.general
community.docker
ansible.posix

Keeping that in code is simpler and safer than hoping the README stays in sync.

What `group_vars/all` Still Does

Very little, by design.

It now only holds shared defaults that are still useful for the active code:

desktop_user
ansible_python_interpreter

Anything more specific belongs in host vars or role defaults.

Storage Model

Storage was simplified on purpose, but Ansible does not replace the whole system /etc/fstab anymore. The OS installer should keep owning /, /boot, EFI and swap because their UUIDs are created during installation.

The repo ships only the data-disk entries:

source: files/serverannah/etc/fstab
deployed into: a marked Ansible block inside /etc/fstab

The server role then:

inserts or updates the marked data-mount block
creates the mount points declared in that source file
runs mount -a

This keeps data mounts versioned without hard-coding boot/root UUIDs that belong to one specific installation.

Disaster Recovery

Backups are Borg (see roles/server/tasks/backup.yml): a daily systemd timer dumps the databases and creates an archive of the /opt/* service dirs into an encrypted repo on the backup disk.

Repo: /mnt/back-up/borg/serverannah (encryption repokey — the key lives in the repo, unlocked by the passphrase, so the repo + passphrase are all you need to restore anywhere).
Passphrase: /etc/ansible/secrets/borg-passphrase, generated on the host, never in this repo. Losing it means losing every archive — back it up separately (a password manager).

The restore path has been tested on a fresh Ubuntu VM (a full archive extracts cleanly: every file decrypts, DB dumps intact). To restore on a new machine you only need borgbackup, the repo (copy the disk, or stream it), and the passphrase:

export BORG_PASSPHRASE='…'
borg list  /path/to/repo                       # pick an archive
borg extract /path/to/repo::serverannah-<ts>   # restores ./opt/... and ./var/...

borg's standalone binary (GitHub releases, borg-linux-glibc236) needs no root, so a restore works even on a machine where you cannot install packages. Match the binary's major version to the one that wrote the repo (currently 1.4.x).

Dotfiles Model

Dotfiles are handled with:

Ansible as orchestrator
Stow as symlink engine for plaintext configs
ansible-vault + a copy-decrypt step for the few configs that contain secrets

Plaintext packages live under files/dotfiles/<package>/ mirroring the home layout. The exact set applied to a machine is whatever its dotfiles_stow_packages lists (see the host's host_vars/ file) — that list is the single source of truth, so this README does not re-enumerate it. Every package directory there is symlinked into $HOME by Stow.

Stow runs with --no-folding on purpose. By default Stow "folds" a config directory that does not yet exist into a single directory symlink pointing into the repo (e.g. ~/.config/gh -> repo/.../gh). Because our repo is a public checkout managed in place, any file an app then writes into that dir (gh's hosts.yml token, ssh known_hosts, nvim lock files) would be written straight into the git tree. --no-folding keeps every directory a real local dir and symlinks only the curated files, so app-written files stay local. The pre-stow "preserve" step is directory-aware to match (it leaves an already-managed dir alone instead of rebuilding it every run, which keeps pulls idempotent).

Do not vendor machine-specific symlinks. Omarchy points some configs at the active theme (for example nvim lua/plugins/theme.lua or btop theme files) via absolute symlinks into its local theme state. Those are gitignored / left out; omarchy recreates them locally, and Stow --no-folding would (correctly) refuse them.

Note two deliberate boundaries:

gh ships only config.yml. hosts.yml holds a GitHub OAuth token and is not committed, even encrypted — let gh auth login recreate it per host.
the ssh package ships only the sanitized public config, which does nothing but Include ~/.ssh/config.local. The real host aliases live in that local file.

Secret dotfiles (currently ~/.ssh/config.local) are stored vault-encrypted under files/dotfiles-secret/ and listed in dotfiles_secret_files. They cannot be Stow-symlinked, because the symlink would hand the application the ciphertext; instead Ansible copies them into place and decrypts on the way (see Secrets And Vault). To update one:

ansible-vault edit --vault-password-file ~/secret.txt files/dotfiles-secret/ssh/config.local

Editing a managed dotfile. After a pull the live file (e.g. ~/.config/espanso/match/base.yml) is a symlink into /opt/ansible-pull, the root-owned pull checkout — so editing it in place fails as read-only (E166 in nvim). That is by design. Always edit the source in your dev clone, then push and pull:

nvim ~/Documents/ansible-autoconfig/files/dotfiles/<pkg>/<path>
cd ~/Documents/ansible-autoconfig && git commit -am "..." && git push
sudo ansible-pull -d /opt/ansible-pull -U https://github.com/djspatule/ansible-autoconfig.git -i hosts local.yml --vault-password-file ~/secret.txt

(Never sudo-edit the /opt/ansible-pull copy: it is untracked and the next pull overwrites it.)

Espanso is managed here too (package espanso, config under files/dotfiles/espanso/). The binary is a special case: it is not in the Arch official repos, so the workstation role builds espanso-wayland from the AUR with yay, as the desktop user, guarded so it only compiles when the binary is missing (a one-time cost on a fresh machine). Because the AUR helper calls pacman, an unattended from-scratch build needs the desktop user to sudo without a prompt; on a single-user Omarchy box that is the norm. Otherwise the build is the one step that needs a hand — run yay -S espanso-wayland once and the next pull skips it. (secret.txt is the vault password, not a sudo password, and would not help here anyway — the prompt comes from yay's internal pacman call, not from Ansible's become.)

GUI/Omarchy desktop config beyond the migrated packages (parts of Hyprland, theme-specific bits) stays out for now; the workstation role is otherwise CLI/TUI focused.

Pi-hole Source Of Truth

Pi-hole policy import now reads from:

files/pihole/policy/adlists_enabled.txt
files/pihole/policy/allowlist_exact.txt

Raw backups are not meant to be the long-term source of truth.

Serverannah Notes

serverannah is the current production target.

Important host-specific values live in host_vars/serverannah, including:

enabled services
public domains
Pi-hole mode
hardware-specific device paths
backup source paths still needed for migration

Conventions For Future Changes

package lists should stay in role defaults
hostnames, domains, and paths should stay in host vars
static config files should go under files/
comments should explain why a choice exists, not narrate every line

Known Reality

the old raspi still matters as migration knowledge, but should not dictate the final design
public DNS and NAT still need deliberate validation before full production cutover
ansible-pull may print a hostname-pattern warning during its internal git step; the actual playbook result matters more than that warning

Navigation Menu

Repository files navigation

Ansible Autoconfig

Usage

ToDo

Guidelines for coding agents

Objectives

Roles

Extra instructions

Do not do

Agent written readme

Design Rules

Active Layout

Current Entry Points

Secrets And Vault

Linting

Testing On A VM

Why `requirements.yml` Stays

What `group_vars/all` Still Does

Storage Model

Disaster Recovery

Dotfiles Model

Pi-hole Source Of Truth

Serverannah Notes

Conventions For Future Changes

Known Reality

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
docs/orchestrator		docs/orchestrator
files		files
group_vars		group_vars
host_vars		host_vars
roles		roles
scripts		scripts
.ansible-lint		.ansible-lint
.gitignore		.gitignore
.pre-commit-config.yaml		.pre-commit-config.yaml
.yamllint		.yamllint
Dinnizer app todo.csv		Dinnizer app todo.csv
README.md		README.md
ansible.cfg		ansible.cfg
hosts		hosts
local.yml		local.yml
requirements.yml		requirements.yml

Latest commit

History

Folders and files

Repository files navigation

Ansible Autoconfig

Usage

ToDo

Guidelines for coding agents

Objectives

Roles

Extra instructions

Do not do

Agent written readme

Design Rules

Active Layout

Current Entry Points

Secrets And Vault

Linting

Testing On A VM

Why requirements.yml Stays

What group_vars/all Still Does

Storage Model

Disaster Recovery

Dotfiles Model

Pi-hole Source Of Truth

Serverannah Notes

Conventions For Future Changes

Known Reality

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Why `requirements.yml` Stays

What `group_vars/all` Still Does