Skip to content

Latest commit

 

History

History
1142 lines (831 loc) · 55.7 KB

File metadata and controls

1142 lines (831 loc) · 55.7 KB

CLAUDE.md

This file provides guidance to Claude Code when working with the Indiekit Cloudron app.

Project Overview

This is a Cloudron-packaged version of Indiekit (IndieWeb server) combined with an Eleventy static site generator. The app runs three processes: Indiekit (Node.js), Eleventy (file watcher), and nginx (static file serving/proxying).

Target Site: https://rmendes.net

User-Only Operations

npm publish requires OTP: The user must run npm publish manually because it requires a one-time password from their authenticator app. Claude cannot complete this step.

CRITICAL: Eleventy Theme Submodule

The eleventy-site/ directory is a Git submodule pointing to the separate theme repository:

Submodule Sync Workflow

When the theme repo is updated, you MUST update this repo's submodule reference:

# Pull latest theme changes into submodule
git submodule update --remote eleventy-site

# Commit the submodule pointer update
git add eleventy-site
git commit -m "chore: update eleventy-site submodule"
git push origin main

# Rebuild and deploy
cloudron build --no-cache
cloudron update --app rmendes.net

Working on Theme Changes

If you need to modify the Eleventy theme:

  1. Work in the theme repo (/home/rick/code/indiekit-dev/indiekit-eleventy-theme)
  2. Commit and push changes there
  3. Return to this repo and update the submodule (commands above)
  4. Rebuild and deploy

Checking Submodule Status

# See which commit the submodule points to
git submodule status

# Check if submodule is behind remote
cd eleventy-site && git fetch && git log HEAD..origin/main --oneline

Common Submodule Issues

Submodule shows as "modified" but you didn't change it:

# Reset submodule to committed state
git submodule update --init

Theme changes not appearing on live site:

  • Did you update the submodule reference in THIS repo?
  • Did you push THIS repo after updating the submodule?
  • Did you rebuild with cloudron build --no-cache?

CRITICAL: Multi-Site Plugin Registry Architecture

This deployment uses a plugin registry + per-site manifest system for multi-site support:

How It Works

Three layers of configuration:

  1. plugin-registry (git submodule)

    • Central version-pinned catalog (plugin-registry/plugin-registry.yaml)
    • Tiers: core (always loaded), post_types, syndicators, endpoints
    • Each entry specifies: package name, version, flags (overridden: true, library: true, default_enabled)
  2. Per-site plugin manifest (sites/<site>/config/plugins.yaml)

    • Enable/disable non-core plugins per site
    • Only lists entries that differ from registry defaults
    • core tier is implicit (always loaded, cannot disable)
  3. Composed outputs (sites/<site>/.compiled/)

    • Generated by make compose SITE=<site> (runs scripts/compose-site.mjs)
    • Merges registry + site manifest → package.json, indiekit.config.js, plugin-loadout.json
    • Dockerfile consumes .compiled/ via --build-arg SITE=<site>

The Plugin Registry

Key entries:

core:
  - key: site-config
    package: "@rmdes/indiekit-endpoint-site-config"     # runtime plugin, per-site identity/branding/homepage
    version: "^1.0.0-beta.6"
  - key: startup-gate
    package: "@rmdes/indiekit-startup-gate"             # library, defers plugin background tasks
    library: true
  - key: auth
    package: "@indiekit/endpoint-auth"
    overridden: true                                     # npm overrides in package.json swaps to @rmdes fork
  # ... other core entries ...

post_types:
  - key: event
    package: "@indiekit/post-type-event"
    default_enabled: false                               # off by default, enable per-site

endpoints:
  - key: activitypub
    package: "@rmdes/indiekit-endpoint-activitypub"
    version: "^3.13.8"
    default_enabled: false                               # off by default, enable per-site

Important flags:

  • overridden: true — npm overrides field swaps this package to @rmdes fork; version is in package.json, NOT registry
  • library: true — installed but never listed in plugins array (imported by other plugins)
  • default_enabled — whether this plugin loads if not mentioned in sites/<site>/config/plugins.yaml

Per-Site Manifests

sites/rmendes/config/plugins.yaml (fuller feature set):

post_types:
  event: { enabled: true }
  jam: { enabled: true }

endpoints:
  activitypub: { enabled: true }
  microsub: { enabled: true }
  conversations: { enabled: true }
  # ... many more ...

sites/chardonsbleus/config/plugins.yaml (minimal, focused):

post_types:
  event: { enabled: false }

endpoints:
  donation: { enabled: true }       # Stripe-backed campaigns
  activitypub: { enabled: false }   # no fediverse for chardonsbleus
  microsub: { enabled: false }      # no social reader for chardonsbleus

Config File System (for per-site overrides)

The sites/<site>/config/ directory contains non-plugin config per site:

File Purpose
plugins.yaml Enable/disable plugins (merged with registry)
indiekit.config.js Site-specific Indiekit config (auth, syndication, integrations)
nginx.conf nginx routes and site identity
env.sh Environment variables (API keys, secrets)
redirects.map, old-blog-redirects.map URL rewrite rules

These are gitignored (encrypted/private per site). When deploying:

  1. make compose SITE=<site> merges registry + site manifest
  2. make prepare materializes root config files from sites/<site>/config/ (or .template fallbacks)
  3. cloudron build --build-arg SITE=<site> bakes the per-site config into the image

Plugin Update Workflow

For registry-managed plugins (everything except overridden: true):

  1. Edit plugin in its standalone repo, bump package.json version, commit/push
  2. User runs npm publish (requires OTP)
  3. Bump version: in plugin-registry/plugin-registry.yaml
  4. Run node scripts/validate.mjs (validates registry syntax)
  5. Commit and push the plugin-registry repo
  6. In indiekit-cloudron: make registry-update (pulls registry submodule, commits pointer)
  7. Deploy: make deploy SITE=<site> APP=<app> (or just make compose and cloudron build if no other changes)

For overridden default plugins (auth, posts, micropub, syndicate, files, share, frontend):

  1. Edit plugin in its standalone repo, bump package.json version, commit/push
  2. User runs npm publish (requires OTP)
  3. Bump version in package.json overrides field (not the registry)
  4. Deploy: make deploy SITE=<site> APP=<app>

Theme: One Canonical Theme — Per-Site Theme Overrides Are NOT Supported

make prepare uses the eleventy-site/ submodule as-is and prints: (using submodule as-is — per-site theme variants are not supported; use siteConfig MongoDB instead).

Per the v2 design acceptance criterion: "One canonical Eleventy theme used by every deployment. No per-site theme forks." All per-site visual/identity variance comes from the @rmdes/indiekit-endpoint-site-config plugin at runtime (MongoDB siteConfig → rendered theme.css, site-config.json, homepage.json).

Legacy: the repo-root overrides/eleventy-site/ directory and sites/<site>/overrides/eleventy-site/ are remnants of the old single-site override mechanism. make prepare does NOT apply them anymore. Do not add files there expecting them to take effect — change the theme in indiekit-eleventy-theme/ (for all sites) or use the site-config admin UI (per site).

Historical trap (why overrides were removed): when a _data/*.js file in the theme became dynamic (reading from a plugin JSON file), a stale static override copied on top of it made the site render old data forever. The siteConfig-based design eliminates this class of bug.

CRITICAL: preset-eleventy Fork

ALWAYS use @rmdes/indiekit-preset-eleventy, NEVER @indiekit/preset-eleventy!

The fork converts the Indiekit url property to an Eleventy permalink property in post frontmatter, ensuring Eleventy generates pages at the canonical Indiekit URLs. The Eleventy data cascade (_data/eleventyComputed.js) adds permalink for existing posts that don't have it.

How it works:

  • Indiekit stores URLs as: /likes/2026/01/30/slug
  • Preset converts url to permalink: /likes/2026/01/30/slug/ in frontmatter
  • Eleventy generates HTML at: /likes/2026/01/30/slug/index.html (from permalink)
  • nginx serves the file directly (no rewrite needed)
  • For old /content/ URLs, nginx redirects (301) to the clean URL
  • Result: Posts are accessible at their canonical Indiekit URLs

The preset also preserves the original Indiekit URL as mpUrl for Micropub edit links in the admin UI.

History: Beta.37 (Feb 2026) changed from deleting url (file-path-based generation) to setting permalink (explicit URL control). This eliminated the URL dualism problem where /notes/2026/02/22/slug redirected to /content/notes/2026-02-22-slug/ instead of serving directly.

CRITICAL: nginx Configuration

Media File Serving

nginx MUST serve media files directly (not proxy to Indiekit):

# Serve media files directly from filesystem
location ~ "^/media/(photos|images|videos|audio)/(.+)$" {
    alias /app/data/content/media/$1/$2;
    expires 30d;
    add_header Cache-Control "public, immutable";
}

# Media API (upload/delete) - proxied to Indiekit
location /media {
    proxy_pass http://127.0.0.1:8080;
    # ... proxy headers
}

CRITICAL: Do NOT include media in the static asset caching regex. The caching regex (location ~* ^/(css|js|fonts|og|pagefind|img|images|graph)/...) MUST NOT list media because nginx regex locations match in order of appearance (first wins). If media is in the caching regex, it intercepts /media/photos/... requests before the alias block above, serving from root /app/data/site (404) instead of alias /app/data/content/media/ (correct). This bug was introduced in 6905ac4 and fixed in 8d001e8 (Mar 2026).

URL Redirects for Post Types

Legacy /content/ URLs redirect (301) to canonical Indiekit URLs:

# Legacy /content/TYPE/YYYY-MM-DD-slug/ → /TYPE/YYYY/MM/DD/slug/
rewrite "^/content/articles/(\d{4})-(\d{2})-(\d{2})-(.+?)/?$" "/articles/$1/$2/$3/$4/" permanent;
rewrite "^/content/notes/(\d{4})-(\d{2})-(\d{2})-(.+?)/?$" "/notes/$1/$2/$3/$4/" permanent;
rewrite "^/content/likes/(\d{4})-(\d{2})-(\d{2})-(.+?)/?$" "/likes/$1/$2/$3/$4/" permanent;
rewrite "^/content/reposts/(\d{4})-(\d{2})-(\d{2})-(.+?)/?$" "/reposts/$1/$2/$3/$4/" permanent;
rewrite "^/content/photos/(\d{4})-(\d{2})-(\d{2})-(.+?)/?$" "/photos/$1/$2/$3/$4/" permanent;
rewrite "^/content/replies/(\d{4})-(\d{2})-(\d{2})-(.+?)/?$" "/replies/$1/$2/$3/$4/" permanent;
rewrite "^/content/bookmarks/(\d{4})-(\d{2})-(\d{2})-(.+?)/?$" "/bookmarks/$1/$2/$3/$4/" permanent;
rewrite "^/content/videos/(\d{4})-(\d{2})-(\d{2})-(.+?)/?$" "/videos/$1/$2/$3/$4/" permanent;
rewrite "^/content/audio/(\d{4})-(\d{2})-(\d{2})-(.+?)/?$" "/audio/$1/$2/$3/$4/" permanent;
rewrite "^/content/jams/(\d{4})-(\d{2})-(\d{2})-(.+?)/?$" "/jams/$1/$2/$3/$4/" permanent;
rewrite "^/content/rsvps/(\d{4})-(\d{2})-(\d{2})-(.+?)/?$" "/rsvps/$1/$2/$3/$4/" permanent;
rewrite "^/content/events/(\d{4})-(\d{2})-(\d{2})-(.+?)/?$" "/events/$1/$2/$3/$4/" permanent;

Markdown for Agents (content negotiation + robots)

Serves clean Markdown to AI agents (GEO/AEO). Full reference: documentation-central/docs/2026-07-01-agent-readable-stack.md. The .md/llms.txt files are generated by the theme build (eleventy-site/lib/markdown-agents.mjs); this repo owns only nginx negotiation + the robots policy.

nginx — keep these blocks in sync across nginx.conf.template AND both sites/<site>/config/nginx.conf:

# .md for articles + notes (deep paths); exact-match twins win over the 406 regex
location ~ "^/(articles|notes)/(.+)\.md$" { try_files /$1/$2/index.md =404; default_type "text/markdown; charset=utf-8"; add_header X-Robots-Tag "noindex" always; ... }
location = /about.md { try_files /about/index.md =404; ... }   # synthesized About twin
location = /index.md { try_files /index.md =404; ... }         # synthesized homepage twin
location ~ "^/(?!articles/|notes/)(.+)\.md$" { return 406 ...; } # other types → clear error
# inside location / :
if ($wants_markdown) {   # $wants_markdown set from Accept: text/markdown
    rewrite "^(/(?:articles|notes)/.+?)/?$" "$1.md" last;
    rewrite "^/about/?$" /about.md last;
    rewrite "^/$"        /index.md last;
}

robots.txt (AI policy) — served by the Cloudron per-app robots.txt UI, NOT by code (Cloudron overrides /robots.txt at the platform proxy; the app's own location = /robots.txt is shadowed). Policy = allow-all with Content-Signal: search=yes, ai-input=yes, ai-train=yes. Canonical source of truth: documentation-central/reference/robots-txt-{rmendes,chardonsbleus}.txt — edit there, then paste into each app's Cloudron UI (instant, no rebuild). It carries a commented, reversible train/retrieve opt-in block.

Verify: make verify-agents URL=https://rmendes.net (homepage markdown, /about.md, /llms.txt, an article .md, robots Content-Signal).

Deploy gotchas (see the reference doc + documentation-central/plans/2026-07-01-llms-txt-implementation.md):

  • The theme's eleventy.after generator is gated on a one-shot flag, not !incremental (the watcher's first full build reports incremental=true).
  • Synthesized /about.md//index.md are only written when the HTML page exists — a site without an About page correctly gets neither (no orphan /about/index.md → no 403).
  • build-status.json is in persistent /app/data, so it shows the PRIOR build's state:ok right after cloudron update — confirm a rebuild by a buildingok transition or advanced output mtime, not the instantaneous value.

Commands

# Build the Cloudron app image
cloudron build

# Build with no cache (REQUIRED after Dockerfile or dependency changes)
cloudron build --no-cache

# Deploy to Cloudron (ALWAYS use --no-backup to skip slow backup step)
cloudron update --app rmendes.net --no-backup

# View logs
cloudron logs -f --app rmendes.net

# SSH into running container
cloudron exec --app rmendes.net

CRITICAL: Always use cloudron build, never docker build directly.

Checking if the Eleventy Build Completed

The atomic release swap is currently DISABLED (see "Build Architecture" below). In start.sh the initial-build-to-new-release block is commented out (INITIAL_BUILD_OK=false, lines ~431–496) because the Eleventy initial build (~3.9 GB peak RSS) + Indiekit exceeds the 4 GB cgroup → OOM. Instead the Eleventy watcher does a full build in place into the current release dir (eleventy --watch --incremental --output=/app/data/site). Consequences: readlink /app/data/site does NOT change after a deploy, there is NO ==> Swapped to release log line, and no new per-deploy releases/ dir is created. Do not wait for a swap — it will never come.

After cloudron update or cloudron restart, the watcher rebuilds /app/data/site in place over ~5 min. cloudron update reporting "App is updated" + a passing health check does NOT mean the new build succeeded — a build can fail and silently leave stale/partial content in place. Confirm the build with these signals:

# 1. Build completion — the authoritative signal (no fatal, all files written)
cloudron logs --app rmendes.net 2>&1 | grep -E "\[11ty\] Wrote [0-9]+ files"

# 2. Build status artifact (start.sh + eleventy.after write it): state ok|building|failed
cloudron exec --app rmendes.net -- cat /app/data/build-status.json

# 3. Any fatal that FAILED the build (a failed build serves stale/partial in place)
cloudron logs --app rmendes.net 2>&1 | grep -iE "Eleventy Fatal Error|Having trouble writing"

# 4. Watcher state: high CPU = still building, low CPU = idle-watching (build done)
cloudron exec --app rmendes.net -- bash -c 'ps -o pcpu,etime,args -C node | grep "eleventy.*--watch"'

# 5. Public smoke test — the ultimate confirmation the new build is live
curl -sL -o /dev/null -w "%{http_code}\n" https://rmendes.net/

Build phases (in order, all inside the watcher's first full build):

  1. OG image generation (~2 min, batch spawning)
  2. Template rendering (3,400+ pages)
  3. Pagefind indexing + eleventy.after hooks (orphan prunes, build-status.json, .indiekit-ready signal)

Timing reference:

  • Warm build (caches populated): ~3-5 min
  • Cold build (empty caches): ~20 min
  • During the in-place build, pages are overwritten progressively — not-yet-rebuilt pages keep serving their previous content (no 404s), but it is NOT a clean atomic swap.

How to tell the build SUCCEEDED:

  • [11ty] Wrote N files (N ≈ full page count) with no Eleventy Fatal Error
  • /app/data/build-status.json shows "state":"ok"
  • The watcher process drops to low CPU (idle-watching)
  • The public URL serves the expected content

Architecture

Directory Structure

Docker Image (read-only at runtime):
├── /app/code/                    # Indiekit core + plugins
│   └── node_modules/             # Indiekit dependencies
├── /app/pkg/
│   ├── eleventy-site/            # Static site generator
│   │   ├── node_modules/         # Eleventy dependencies (NEVER copy to /app/data)
│   │   ├── _includes/            # Nunjucks templates
│   │   ├── _data/                # Site data files
│   │   ├── css/                  # Compiled CSS (Tailwind)
│   │   ├── content -> /app/data/content    # SYMLINK (created in Dockerfile)
│   │   ├── _site -> /app/data/site         # SYMLINK
│   │   ├── .cache -> /app/data/cache       # SYMLINK
│   │   └── uploads -> /app/data/uploads    # SYMLINK
│   ├── start.sh
│   ├── nginx.conf
│   └── indiekit.config.js.template

Runtime (writable, backed up):
├── /app/data/
│   ├── config/                   # indiekit.config.js, env.sh, .secret
│   ├── content/                  # User posts (notes/, articles/, etc.)
│   ├── releases/                 # Eleventy build output dir(s)
│   │   └── 1708400000/           # The single reused release dir (built IN PLACE; not recreated per deploy)
│   ├── site -> releases/1708400000  # SYMLINK; target is rebuilt in place (atomic swap currently disabled)
│   ├── cache/                    # Eleventy cache
│   ├── images/                   # User-uploaded images
│   └── uploads/                  # Media uploads

Process Architecture

  1. nginx (port 3000) - Entry point, serves static files from /app/data/site (symlink), proxies to Indiekit
  2. Eleventy (watcher) - Rebuilds site incrementally when content changes
  3. Indiekit (port 8080) - Handles Micropub, authentication, admin UI
  4. Syndication poller - Background process polling /syndicate every 2 minutes
  5. Webmention sender - Background process polling /webmention-sender every 5 minutes

Build Architecture (in-place watcher; atomic swap currently DISABLED)

The documented zero-downtime atomic-swap flow is retained but commented out in start.sh (lines ~431–496, with INITIAL_BUILD_OK=false hardcoded). It is abandoned, not just memory-blocked — see below.

MEASURED twice 2026-06-20 — re-enabling OOMs even with more RAM; DO NOT re-enable without a deeper fix:

  • At the old 3840 MB cgroup: the swap's separate initial build climbed to ~3839 MB (99.9%) and swap-thrashed under the OOM ceiling — never swapped, never cleanly failed over (13+ min).
  • After raising the app to 5120 MB: the initial build peaked ~3401 MB, then EXPLODED to 5118 MB and cgroup-OOM-killed (kernel OOM, no heap snapshot). It grew to FILL the new memory.

Root cause (why more RAM doesn't help): the initial build has a late-phase memory explosion — pagefind indexing 3,400 pages uses native memory OUTSIDE V8's heap (uncapped by --max-old-space-size) — AND it runs CONCURRENTLY with Indiekit's startup spike (30+ plugins, ActivityPub/Fedify, Mongo). The in-place watcher build avoids both (runs after Indiekit settles). The safe fallback (INITIAL_BUILD_OK=false → in-place watcher) kept the site serving throughout both tests. The "~3,260 MB peak" figures elsewhere in this doc are stale.

To ever revisit the swap: pagefind must run out-of-process / memory-capped AND the build must be deferred until Indiekit settles (at which point it effectively IS the watcher). Not worth it for the zero-downtime gain given the in-place build works. Keep the app at ≥5 GB regardless — it removed the standing OOM risk on the in-place build (now ~67–75% of 5120 MB vs 99.9% at 3840 MB).

Actual current flow on container restart:

nginx starts → /app/data/site → symlink to the SAME (reused) release dir → serves existing content
Indiekit starts → ready on :8080
Eleventy WATCHER starts → eleventy --watch --incremental --output=/app/data/site
  → first pass is a FULL build, written IN PLACE into the current release dir (~5 min)
  → eleventy.after writes build-status.json + creates /app/data/.indiekit-ready
  → then incremental rebuilds on content changes
(no new release dir per deploy; no `mv -T` swap; `readlink /app/data/site` unchanged)

The release dir got its timestamp name from a one-time migration (start.sh ~line 400, converting an old real /app/data/site dir to a symlink) — it is NOT recreated per deploy.

Visitors experience: during the ~5 min in-place full build, already-rebuilt pages show new content and not-yet-rebuilt pages show their previous content (no 404s). NOT an atomic swap — there is a window of mixed old/new pages.

If the build fails: the watcher supervisor captures the crash in /app/data/build-status.json ("state":"failed") and restarts the watcher with exponential backoff. The in-place dir keeps serving whatever was last written (stale/partial) until a build succeeds. This is why a crashing build is a silent stale-serve — always verify build completion (see "Checking if the Eleventy Build Completed").

Rollback: deploys reuse one release dir (no per-deploy timestamped releases), so rollback is by redeploying a previous image (make deploy after checking out the prior code/submodule), NOT by repointing the symlink.

Startup Gate — Plugin Background Task Deferral

All @rmdes/* plugins with background tasks use @rmdes/indiekit-startup-gate to defer heavy work until after Eleventy's first successful build. This prevents plugins from competing for the ~2.8 GB that Eleventy needs during its build.

Signal file lifecycle in start.sh:

start.sh removes /app/data/.indiekit-ready    ← BEFORE Indiekit starts (line ~195)
Indiekit starts → plugins call waitForReady() → file not found → wait
Initial build is DISABLED (INITIAL_BUILD_OK=false) → start.sh skips the swap/signal branch
Eleventy WATCHER starts → does a full in-place build
Eleventy's eleventy.after hook creates the signal  ← the live path
Plugins detect signal → start background tasks

Key implementation details:

  • rm -f /app/data/.indiekit-ready MUST be before the node ... indiekit ... serve line in start.sh
  • Because the initial build + swap branch is disabled, .indiekit-ready is ALWAYS created by eleventy.config.js's eleventy.after hook when the watcher's first full build completes (NOT by start.sh's touch in the disabled swap-success branch)
  • The signal uses !existsSync() guard so it's only created once per boot

When adding a new plugin to the Dockerfile: If the plugin has background tasks, verify it uses @rmdes/indiekit-startup-gate. See workspace CLAUDE.md for the full pattern.

CRITICAL: npm Overrides for Forked Default Plugins

When we fork a default Indiekit plugin (e.g., @indiekit/endpoint-share@rmdes/indiekit-endpoint-share), we use npm overrides in package.json:

{
  "overrides": {
    "@indiekit/endpoint-auth": "npm:@rmdes/indiekit-endpoint-auth@^1.0.0-beta.31",
    "@indiekit/endpoint-share": "npm:@rmdes/indiekit-endpoint-share@^1.0.4"
  }
}

npm installs our fork under the upstream name. When Indiekit's default plugin loader does import("@indiekit/endpoint-share"), it gets our fork's code transparently.

Rules:

  • Add the override to package.json — version MUST be pinned here, NOT in the registry
  • Do NOT list @rmdes/<fork> in indiekit.config.js plugins array — the default plugins already list @indiekit/, and the override swaps it. Adding both creates duplicates
  • Do NOT add overridden: true entries to the Dockerfile COPY npm install — they're transitive deps of @indiekit/indiekit
  • Do NOT patch defaults.js to remove upstream plugins

Currently overridden packages (see package.json for exact versions):

  • @indiekit/endpoint-auth@rmdes/indiekit-endpoint-auth
  • @indiekit/endpoint-posts@rmdes/indiekit-endpoint-posts
  • @indiekit/endpoint-micropub@rmdes/indiekit-endpoint-micropub
  • @indiekit/endpoint-syndicate@rmdes/indiekit-endpoint-syndicate
  • @indiekit/endpoint-files@rmdes/indiekit-endpoint-files
  • @indiekit/endpoint-share@rmdes/indiekit-endpoint-share
  • @indiekit/frontend@rmdes/indiekit-frontend

See also: The plugin-registry marks these with overridden: true so the compose script knows not to add them as direct deps.

Security Hardening

Patches Applied to Upstream

Two files in patches/ are copied over upstream Indiekit files during Docker build:

Patch Target Purpose
patches/routes.js node_modules/@indiekit/indiekit/lib/routes.js Remove rate limiting from authenticated routes (prevents 429 behind reverse proxy)
patches/error.js node_modules/@indiekit/indiekit/lib/middleware/error.js Suppress stack traces in production (prevents info leakage)
patches/indieauth.js node_modules/@indiekit/indiekit/lib/indieauth.js Double-gated devMode auth bypass — requires BOTH devMode: true in config AND INDIEKIT_ALLOW_DEV_AUTH=1 env var. Prevents accidental production exposure if devMode is left enabled.

When upstream Indiekit updates, diff the new files against our patches and re-apply the same principles.

nginx Security Headers

Added in nginx.conf.template (generic) and sites/<site>/config/nginx.conf (per-site):

Header Value Purpose
X-Content-Type-Options nosniff Prevent MIME-type sniffing
X-Frame-Options SAMEORIGIN Prevent clickjacking
Referrer-Policy strict-origin-when-cross-origin Limit referrer leakage
Permissions-Policy camera=(), microphone=(), geolocation=() Disable unused browser APIs
Content-Security-Policy See below Restrict resource loading

CRITICAL: CSP and External Scripts

The Content-Security-Policy header MUST allow https://cdn.jsdelivr.net in script-src and style-src because the Eleventy theme loads Alpine.js and lite-youtube-embed from this CDN.

Current CSP:

default-src 'self';
script-src 'self' 'unsafe-inline' 'unsafe-eval' https://cdn.jsdelivr.net;
style-src 'self' 'unsafe-inline' https://cdn.jsdelivr.net https://fonts.googleapis.com;
img-src 'self' data: https:;
media-src 'self' https:;
font-src 'self' https://fonts.gstatic.com;
frame-src 'self' https://www.youtube.com https://www.youtube-nocookie.com https://open.spotify.com https://embed.acast.com https://player.vimeo.com https://w.soundcloud.com;
connect-src 'self' https: https://cdn.jsdelivr.net;
base-uri 'self';
form-action 'self'

Lesson learned (Alpine.js): The initial CSP (Feb 2026) omitted cdn.jsdelivr.net, which blocked Alpine.js from loading. Without Alpine.js, the FAB (Floating Action Button) didn't initialize, making it appear that login detection was broken. The auth check (admin.js) worked fine — it dispatched the indiekit:auth event — but Alpine.js wasn't loaded to listen for it.

Lesson learned (connect-src): The service worker uses fetch() to cache external avatars (from Mastodon instances like assets.chaos.social, files.mastodon.social, etc.). This is governed by connect-src, not img-src. Without https: in connect-src, avatars fail on first load but appear after hard refresh (which bypasses the service worker).

When adding new CDN dependencies to the theme:

  • Update nginx.conf.template (generic defaults)
  • Update CSP in per-site nginx configs: sites/rmendes/config/nginx.conf, sites/chardonsbleus/config/nginx.conf
  • Run make prepare before building (materializes per-site config to root)

JWT Injection Prevention in start.sh

The syndication and webmention background processes generate JWT tokens. The original code used shell interpolation ('$VARIABLE') to inject values into Node.js code, which could allow injection if the variable values contained special characters.

Fix: Environment variables are passed via JWT_ORIGIN="$VAR" JWT_SECRET="$VAR" node -e "..." and read with process.env.JWT_ORIGIN inside the Node.js code, eliminating shell injection risk.

Memory Tuning

The Cloudron container has a 3.5 GB (3,584 MB) cgroup memory limit shared across all processes (Indiekit, Eleventy, nginx, Redis, background jobs).

CRITICAL: Node.js Heap Caps

Process Heap Cap Set In Why
Indiekit 1536MB start.sh (NODE_OPTIONS="--max-old-space-size=1536") Raised from 768 MB after observing growth in steady-state RSS as more plugins (ActivityPub, Microsub, Conversations) were added. Mar 2026 heap snapshot showed 137 MB; current 30+ plugin load runs ~300 MB RSS. 1536 MB cap leaves ample headroom without crowding Eleventy's 2560 MB watcher.
Eleventy initial build 2048MB start.sh (NODE_OPTIONS="--max-old-space-size=2048") Full build processes all posts, OG images, and Pagefind index
Eleventy watcher 2560MB start.sh (NODE_OPTIONS="--max-old-space-size=2560 --expose-gc --heapsnapshot-signal=SIGUSR2 --diagnostic-dir=/tmp") Watcher's initial full build peaks above 2304 MB V8 heap (3,400+ pages in memory). GC hook returns memory to OS after build.
og-cli 512MB eleventy.config.js (--max-old-space-size=512 --expose-gc) V8 heap only uses ~22 MB; cap is safety margin. WASM native memory is the real consumer (not limited by this flag).

Post-Build GC

eleventy.config.js calls global.gc() in the eleventy.after hook (requires --expose-gc). This forces V8 to release freed heap pages back to the OS via madvise(MADV_DONTNEED). Without it, post-build allocations stay resident because watch mode has no allocation pressure to trigger GC naturally.

The GC hook also logs V8 heap space breakdown and supports HEAP_SNAPSHOT=1 env var to write a snapshot to /tmp for analysis.

Heap Snapshot Analysis (Mar 2026)

Heap snapshots taken with V8 HeapProfiler reveal the actual memory consumers:

Indiekit process (PID 33): 137 MB heap, ~300 MB RSS

  • Strings: 39 MB (plugin source code, template strings)
  • Compiled code: 35 MB (30+ plugins)
  • Native buffers: 19 MB (TLS/crypto)
  • Arrays: 19 MB
  • Verdict: healthy, no leaks

Eleventy watcher: 1,127 MB heap after GC, ~1,600 MB RSS

V8 Heap Space Size What
large_object_space 707 MB Rendered HTML pages retained for incremental rebuild diffing
old_space 408 MB Collections, template data, Eleventy internal structures
code_space 8 MB Compiled JavaScript
trusted_space 3 MB V8 internal

Root cause of the 707 MB large_object_space: Eleventy's --watch --incremental mode retains all 3,070 rendered HTML pages in memory. Each complete HTML page (base layout + sidebar + post content + Tailwind utility classes + microformat markup) averages 228 KB. This is not a leak — it's Eleventy's watch-mode architecture.

Breakdown of retained large strings:

  • 3,070 full HTML pages (<!DOCTYPE html>...): 682 MB
  • 426 collection/feed pages: 36 MB
  • 42 category JSON feeds: 8 MB
  • 37 category RSS feeds: 7 MB
  • Module source code: ~6 MB

170 MB of JSArrayBufferData (1,579 native buffers) from eleventy-img's in-memory image cache.

OG Image Generator — Batch Spawning (Fixed Mar 2026)

The OG image generator (lib/og.js) uses Satori (Yoga WASM) for SVG layout and Resvg (Rust WASM) for PNG rendering. These WASM modules allocate native memory outside V8's heap, meaning --max-old-space-size has no effect on them. Without mitigation, native memory grows unbounded with each image processed.

Problem: Single-process OG generation of 2,350+ images peaked at 2,976 MB RSS — V8 heap was only 23 MB, the rest was WASM native allocations. During watcher rebuilds (watcher already at ~1.8 GB), this exceeded the cgroup limit and OOM-killed the process.

Solution: Batch spawning. eleventy.config.js spawns og-cli in a loop with batchSize=100. Each invocation generates up to 100 images, saves the manifest, and exits with code 2 ("more remain"). The spawner catches exit code 2 and re-spawns. When og-cli exits, the OS reclaims ALL memory — both V8 heap and WASM native allocations. Exit code 0 = all done.

Measured results (Mar 2026, full regeneration during watcher rebuild):

24 batches × 100 images + 1 final batch of 54 = 2,454 images generated
Per-batch peak RSS: 452-469 MB (flat across all batches, no growth)
V8 heap: 17-23 MB per batch
Total time: ~2 minutes
Container cgroup during generation: ~2,300 MB (watcher 1.8 GB + og-cli batch ~460 MB)

Additional safeguards:

  • global.gc() every 5 images within each batch (reclaims JS wrappers referencing WASM objects)
  • Manifest saved every 10 images (preserves progress if OOM-killed mid-batch)
  • newManifest seeded from existing manifest (unscanned entries survive batch writes)

Memory Budget (Measured Mar 2026)

Process Steady State Peak (during build) Notes
Indiekit ~300 MB ~300 MB Stable, no leaks
Eleventy watcher ~1,800 MB ~2,800 MB 1,140 MB heap steady state; peaks at ~2,560 MB V8 heap during initial build (3,400+ pages)
og-cli (per batch) not running ~460 MB 100 images/batch, fresh process each, flat memory
nginx ~15 MB ~15 MB 2 worker processes
Redis ~12 MB ~12 MB Fedify KV + plugin cache

Peak memory during OG generation: watcher (~2,800 MB) + og-cli batch (~460 MB) = ~3,260 MB, fitting within the 3,584 MB limit. After all batches complete and GC runs, steady state is ~2,641 MB / 3,584 MB (74% utilization).

On-Demand Heap Snapshots

Two mechanisms for heap analysis:

  1. SIGUSR2: Send kill -USR2 <watcher_pid> — V8 writes snapshot to /tmp/ (via --heapsnapshot-signal=SIGUSR2 --diagnostic-dir=/tmp)
  2. HEAP_SNAPSHOT=1: Set env var before a build — the GC hook writes snapshot after build completes

Analysis script: use node to parse the JSON snapshot file and aggregate by type/size.

Memory Monitor

A background process in start.sh logs RSS + swap for Indiekit and Eleventy every 10 minutes:

[mem-monitor] indiekit=318188kB+423384kBswap eleventy=1573160kBswap cgroup=2373MB

Optimization Opportunities

Potential further reductions within the 3.5 GB limit:

  1. Reduce per-category feeds: Each category generates feed.json + feed.xml (~200 extra pages, ~45 MB)
  2. Reduce per-page HTML size: Stripping sidebar from non-content pages would save ~50 KB/page
  3. Constrain og-cli: FIXED — batch spawning (100 images/invocation) keeps peak at ~460 MB per batch
  4. Increase container to 4 GB: DONE — container raised from 3 GB to 3.5 GB (Mar 2026). Steady state ~2,641 MB / 3,584 MB (74%)

Redis for Fedify KV Store (MANDATORY)

As of AP plugin 2.2.0, the Fedify KV store and plugin cache use Redis instead of MongoDB's ap_kv collection. This was the primary OOM fix — the ap_kv collection grew unbounded (~14K entries/day) and consumed ~50MB+ in MongoDB, causing memory pressure.

Redis provides native TTL support so idempotence keys and cache entries auto-expire. The Fedify KV store uses fedify:: key prefix, the plugin cache uses indiekit: key prefix.

If Redis is unavailable, the AP plugin falls back to in-memory storage, which loses state on restart and eventually causes memory growth.

nginx Worker Processes

Set to worker_processes 2 (not auto which spawns 10 on a 10-core machine). For a personal site, 2 workers are more than sufficient and save ~80MB vs 10 workers.

Critical Patterns (MUST FOLLOW)

1. Symlinks MUST Be Created in Dockerfile

The /app/pkg filesystem is read-only at runtime. Symlinks cannot be created in start.sh.

# CORRECT - Create symlinks in Dockerfile (dangling during build, valid at runtime)
RUN rm -rf /app/pkg/eleventy-site/content && ln -s /app/data/content /app/pkg/eleventy-site/content && \
    rm -rf /app/pkg/eleventy-site/_site && ln -s /app/data/site /app/pkg/eleventy-site/_site && \
    rm -rf /app/pkg/eleventy-site/.cache && ln -s /app/data/cache /app/pkg/eleventy-site/.cache
# WRONG - This fails at runtime
ln -sf /app/data/content /app/pkg/eleventy-site/content  # Read-only file system error

2. NEVER Copy node_modules to /app/data

Copying node_modules to /app/data causes:

  • Massive backup sizes (100MB+ for each backup)
  • Slow deployments
  • Wasted storage
# WRONG - node_modules gets backed up
cp -r /app/pkg/eleventy-site/* /app/data/eleventy/

# CORRECT - Run from /app/pkg where node_modules already exists
cd /app/pkg/eleventy-site
./node_modules/.bin/eleventy

3. NODE_ENV Timing

Set NODE_ENV=production AFTER npm install and builds, not before:

# CORRECT
RUN npm install                    # Gets all dependencies including devDependencies
RUN ./node_modules/.bin/tailwindcss -i css/tailwind.css -o css/style.css --minify
ENV NODE_ENV=production           # Set AFTER builds complete

# WRONG - devDependencies won't install
ENV NODE_ENV=production
RUN npm install                    # Tailwind not installed!

4. ESM Modules Configuration

The Eleventy site uses ESM modules. Both files must be consistent:

package.json must have:

{
  "type": "module"
}

eleventy.config.js must use ESM syntax:

import pluginWebmentions from "@chrisburnell/eleventy-cache-webmentions";
import { feedPlugin } from "@11ty/eleventy-plugin-rss";

export default function (eleventyConfig) {
  // ...
}

5. Disable Markdown Template Engine

Markdown content may contain code samples with {{ syntax. Prevent Nunjucks from parsing markdown:

// eleventy.config.js
return {
  markdownTemplateEngine: false,  // CRITICAL - prevents parsing {{ in markdown
  htmlTemplateEngine: "njk",
};

6. Cache Busting for Clean Builds

After changing Dockerfile or dependencies, increment CACHE_BUST:

# Increment to force rebuild — read the current value from the Dockerfile
ARG CACHE_BUST=N  # Bump to N+1

Then run: cloudron build --no-cache

The current value lives in Dockerfile at the top (line ~4). Always read it before bumping rather than assuming a specific number.

7. Eleventy Collection Paths

Collections use paths relative to Eleventy's input directory. The symlink makes content appear at content/:

// CORRECT - single content/ prefix
eleventyConfig.addCollection("posts", function (collectionApi) {
  return collectionApi.getFilteredByGlob("content/**/*.md");
});

// WRONG - double content/content/
eleventyConfig.addCollection("posts", function (collectionApi) {
  return collectionApi.getFilteredByGlob("content/content/**/*.md");
});

8. start.sh Must Run from /app/pkg

# CORRECT - run from where node_modules exists
cd /app/pkg/eleventy-site
gosu cloudron:cloudron ./node_modules/.bin/eleventy --output=/app/data/site

# WRONG - path doesn't exist
gosu cloudron:cloudron /app/code/node_modules/.bin/eleventy

9. Data Files Must Use ESM Syntax

All _data/*.js files must use ESM exports when package.json has "type": "module":

// CORRECT - ESM syntax
export default {
  name: process.env.SITE_NAME || "My Site",
  url: process.env.SITE_URL || "https://example.com",
};

// WRONG - CommonJS syntax (causes "module is not defined" error)
module.exports = {
  name: process.env.SITE_NAME || "My Site",
};

10. Atomic Release Swap for Zero-Downtime Builds (currently DISABLED — retained for re-enable)

NOT active. This atomic-swap pattern is commented out in start.sh (INITIAL_BUILD_OK=false) because the initial Eleventy build OOMs alongside Indiekit in the 4 GB cgroup. The live path is an in-place watcher build (see "Build Architecture"). The pattern below is the swap design to restore if the memory budget ever allows.

The (disabled) design builds to a timestamped release directory, then atomically swaps the symlink so the old site serves throughout the build — no 404s during restart.

# Build to new release directory (old site still serving)
RELEASE_TS=$(date +%s)
NEW_RELEASE="/app/data/releases/${RELEASE_TS}"
mkdir -p "${NEW_RELEASE}"
gosu cloudron:cloudron ./node_modules/.bin/eleventy --output="${NEW_RELEASE}"

# Atomic swap: create temp symlink, rename over current (rename(2) is atomic)
ln -s "${NEW_RELEASE}" /app/data/site_tmp
mv -T /app/data/site_tmp /app/data/site
nginx -s reload

# Cleanup: keep only 2 most recent releases for rollback
cd /app/data/releases && ls -1t | tail -n +3 | xargs -r rm -rf

Key details:

  • mv -T is an atomic rename(2) syscall — the symlink is never missing
  • ln -snf is NOT atomic (it unlinks then links, creating a gap)
  • The watcher uses --watch --incremental to only rebuild changed pages
  • Hooks (OG images, Pagefind, WebSub) are skipped during incremental rebuilds
  • Why disabled: running the full build as a separate process before Indiekit-concurrent steady state still peaked over the cgroup limit; the watcher-only in-place build avoids a second concurrent Eleventy process. Re-enabling needs either a larger cgroup or a lower build-time heap.

11. Ignore Output Directory in Eleventy Config

Prevent Eleventy from processing files in the output directory (which is a symlink):

// eleventy.config.js
eleventyConfig.ignores.add("_site");
eleventyConfig.ignores.add("_site/**");

Eleventy Performance Optimizations (Mar 2026)

The theme includes several performance optimizations that dramatically reduce incremental rebuild times:

Data File Caching (lib/data-fetch.js)

A shared cachedFetch helper wraps @11ty/eleventy-fetch with:

  • Watch-mode cache extension: During ELEVENTY_RUN_MODE !== "build", cache duration extends to 4 hours (vs 5-15 min default). This prevents 13 network data files from re-fetching APIs on every incremental rebuild.
  • AbortController timeout: 10-second hard timeout on all network requests to prevent slow APIs from hanging the build.

Result: Data File phase went from 12,169ms → 28ms on incremental rebuilds (99.8% reduction).

Filter Memoization (eleventy.config.js)

Nunjucks filters called thousands of times per build are memoized with Map caches cleared on eleventy.before:

  • dateDisplay, date, isoDate — date formatting
  • hash — MD5 file hashing for cache busting
  • aiPosts, aiStats — computed data

html-transformer Pre-Check

The default @11ty/eleventy/html-transformer transform is overridden with a pre-check that skips the full PostHTML parse/serialize cycle (~3ms/page) for pages without <img> tags.

Build Time Reference

Build Type Time Pages Notes
Cold build (empty caches) ~20 min 3,400+ First deploy or after wiping .cache/. Regenerates all 2,400+ OG images, fetches all unfurl URLs, all API data files
Warm build (caches populated) ~3 min 3,400+ Normal container restart. OG manifest skips existing images, unfurl/data caches hit disk
Incremental rebuild (watcher) ~25s 1,047 written, 2,392 skipped Triggered by content changes. Data files cached 4h in watch mode

What makes a build "cold": The OG manifest (.cache/og/manifest.json), unfurl cache (.cache/unfurl/), and eleventy-fetch cache (.cache/eleventy-fetch/) are empty. This happens on first deploy or if /app/data/cache/ is wiped. The symlink .cache → /app/data/cache persists these across container restarts, so normal restarts are warm builds.

What makes a build "warm": Caches are populated from a previous build. OG generation only processes new/changed posts (manifest-based diffing). Unfurl URLs and API data are served from disk cache. The dominant cost is template rendering + Pagefind indexing (~2-3 min).

Eleventy Site Configuration

Required Plugins

The site depends on these plugins (all in package.json):

  • @11ty/eleventy - Core
  • @11ty/eleventy-plugin-rss - RSS feed generation
  • @11ty/eleventy-img - Image optimization
  • @chrisburnell/eleventy-cache-webmentions - Webmentions
  • eleventy-plugin-embed-everything - Auto-embed social posts
  • @quasibit/eleventy-plugin-sitemap - Sitemap generation

Content Collections

Collection Path Description
posts content/**/*.md All content combined
notes content/notes/**/*.md Short posts
articles content/articles/**/*.md Long-form articles
bookmarks content/bookmarks/**/*.md Saved links
photos content/photos/**/*.md Photo posts
likes content/likes/**/*.md Liked content
feed content/**/*.md (limit 20) RSS feed

Pagination Configuration

Pagination must NOT use reverse: true because collections are already sorted newest-first:

pagination:
  data: collections.notes
  size: 20
  alias: paginatedNotes
  # NO reverse: true - collections already sorted by date descending

Debugging

Common Errors

"Blog coming soon" placeholder:

  • Eleventy build failed
  • Check: cloudron logs -f --app rmendes.net
  • Look for: template errors, missing modules, path issues

"module is not defined in ES module scope":

  • package.json missing "type": "module" or config using CommonJS syntax
  • Fix: Ensure both package.json and config use ESM consistently

"unexpected token: /" in templates:

  • markdownTemplateEngine: "njk" is parsing code in markdown
  • Fix: Set markdownTemplateEngine: false

"Cannot find module":

  • Wrong path to node_modules
  • Fix: Run from directory where npm install was executed

Massive backup sizes:

  • node_modules in /app/data
  • Fix: Remove any code that copies to /app/data, run from /app/pkg

Useful Debug Commands

# Check what's in /app/data (should NOT have node_modules)
cloudron exec --app rmendes.net -- ls -la /app/data/

# Check symlinks are correct
cloudron exec --app rmendes.net -- ls -la /app/pkg/eleventy-site/

# Manual Eleventy build
cloudron exec --app rmendes.net -- bash -c "cd /app/pkg/eleventy-site && ./node_modules/.bin/eleventy"

# Check Eleventy version
cloudron exec --app rmendes.net -- bash -c "cd /app/pkg/eleventy-site && ./node_modules/.bin/eleventy --version"

MongoDB Access

IMPORTANT: Use CLOUDRON_MONGODB_URL (NOT MONGODB_URL) to connect to MongoDB in the Cloudron container.

Quick MongoDB Query

# Run a MongoDB query on the live site
cloudron exec --app rmendes.net -- bash -c 'mongosh "$CLOUDRON_MONGODB_URL" --quiet --eval "
  // Your JavaScript query here
  db.collectionName.find({}).limit(5).toArray()
"'

Common Collections

Collection Description
blogrollBlogs Blog entries for the blogroll
blogrollItems Individual posts from blogs
blogrollSources OPML/Microsub sources
microsub_feeds Microsub feed subscriptions
microsub_items Items from Microsub feeds
microsub_channels Microsub channels
posts Indiekit posts

Example Queries

# Count documents in a collection
cloudron exec --app rmendes.net -- bash -c 'mongosh "$CLOUDRON_MONGODB_URL" --quiet --eval "
  db.blogrollBlogs.countDocuments({})
"'

# Find blogs with error status
cloudron exec --app rmendes.net -- bash -c 'mongosh "$CLOUDRON_MONGODB_URL" --quiet --eval "
  db.blogrollBlogs.find({status: \"error\"}).toArray()
"'

# Delete documents matching a query
cloudron exec --app rmendes.net -- bash -c 'mongosh "$CLOUDRON_MONGODB_URL" --quiet --eval "
  db.blogrollBlogs.deleteMany({status: \"error\", microsubFeedId: null})
"'

# Update documents
cloudron exec --app rmendes.net -- bash -c 'mongosh "$CLOUDRON_MONGODB_URL" --quiet --eval "
  db.blogrollBlogs.updateMany({status: \"error\"}, {\\\$set: {status: \"active\"}})
"'

Environment Variables

MongoDB connection details are available via these environment variables:

Variable Description
CLOUDRON_MONGODB_URL Full connection string (use this!)
CLOUDRON_MONGODB_HOST MongoDB hostname (mongodb)
CLOUDRON_MONGODB_PORT MongoDB port (27017)
CLOUDRON_MONGODB_DATABASE Database name
CLOUDRON_MONGODB_USERNAME Username
CLOUDRON_MONGODB_PASSWORD Password

Note: The Indiekit config uses process.env.MONGODB_URL which is mapped from CLOUDRON_MONGODB_URL in the start.sh script.

File Checklist for Changes

When modifying this app, verify:

File Check
Dockerfile Symlinks created with ln -s, NODE_ENV after installs
start.sh Runs from /app/pkg/eleventy-site, in-place watcher build (atomic swap disabled), no cp to /app/data
package.json Has "type": "module"
eleventy.config.js ESM syntax, markdownTemplateEngine: false, ignores _site, correct glob paths
_data/*.js All use export default, not module.exports
nginx.conf Serves from /app/data/site, proxies /admin to port 8080

CRITICAL: Behavioral Rules

NEVER disable, remove, or comment out functionality to "fix" a problem

If something is broken, find and fix the root cause. Do NOT:

  • Disable a plugin/tool to avoid an error it causes
  • Remove a layout/template to avoid a rendering issue
  • Comment out image processing to skip missing-file errors
  • Remove UI elements (sidebar, widgets, navigation) because they have a bug
  • Delete content or features without explicit user approval

If the fix is not clear, investigate deeper or ask the user. Destructive shortcuts are never acceptable.

NEVER edit files in the wrong repo

  • Theme files: Edit in indiekit-eleventy-theme/, NOT in indiekit-cloudron/eleventy-site/ (submodule)
  • Plugin code: Edit in the standalone indiekit-endpoint-*/ or indiekit-syndicator-*/ repos, NOT in indiekit/packages/ (upstream fork)
  • After editing the source, propagate changes (submodule update, npm publish, cloudron build)

Plugin Update Workflow (MUST follow in order)

For registry-managed plugins:

  1. Edit plugin in its standalone repo
  2. Bump version in package.json, commit and push
  3. STOP — tell user to run npm publish (requires OTP)
  4. Wait for user to confirm publish is done
  5. Update plugin-registry/plugin-registry.yaml with new version
  6. Run node plugin-registry/scripts/validate.mjs to validate registry syntax
  7. Commit and push plugin-registry repo
  8. In indiekit-cloudron: make registry-update (pulls submodule, commits pointer)
  9. Deploy per-site: make deploy SITE=rmendes APP=rmendes.net (compose + prepare + build + update)

For overridden default plugins:

  1. Edit plugin in its standalone repo, bump version, push
  2. STOP — tell user to run npm publish (requires OTP)
  3. Wait for user to confirm publish is done
  4. In indiekit-cloudron: update package.json overrides field with new version
  5. Deploy per-site: make deploy SITE=rmendes APP=rmendes.net

Anti-Patterns (NEVER DO)

  1. docker build - Use cloudron build
  2. ❌ Creating symlinks in start.sh - filesystem is read-only at runtime
  3. ❌ Copying anything to /app/data/eleventy/ - bloats backups
  4. ❌ Setting NODE_ENV=production before npm install
  5. ❌ Using reverse: true with already-sorted collections
  6. ❌ Using CommonJS (module.exports) in ESM project ("type": "module")
  7. markdownTemplateEngine: "njk" with code samples in content
  8. ❌ Referencing /app/code/node_modules/.bin/eleventy for Eleventy
  9. ❌ Wiping /app/data/site/* before building — the watcher rebuilds it in place; wiping causes a full-downtime gap (atomic swap is currently disabled, so there's no old-release fallback)
  10. ❌ Using blog.rmendes.net instead of rmendes.net - this is the production domain
  11. ❌ Disabling/removing features to work around bugs - find the root cause
  12. ❌ Editing theme files in eleventy-site/ submodule instead of indiekit-eleventy-theme/
  13. ❌ Editing plugin code in indiekit/packages/ instead of standalone indiekit-endpoint-*/ repos
  14. ❌ Running cloudron build before user confirms npm publish is done
  15. ❌ Publishing to npm without bumping the version first

Data Corruption Recovery

Circular Symlink in /app/data/content

If you see ELOOP: too many symbolic links encountered with paths like content/content/content/..., there's a circular symlink:

# Check for circular symlink
cloudron exec --app rmendes.net -- ls -la /app/data/content/

# If you see: content -> /app/data/content (symlink back to itself), remove it:
cloudron exec --app rmendes.net -- rm /app/data/content/content

This can happen when buggy code copies symlinks instead of following them.

Cleaning Up Old /app/data/eleventy Directory

If /app/data/eleventy/ exists (from buggy old start.sh), remove it:

cloudron exec --app rmendes.net -- rm -rf /app/data/eleventy

Manual Rebuild After Data Cleanup

After fixing data issues, trigger a full rebuild with cloudron restart — it re-runs start.sh, which sources the full environment (env.sh, secrets, plugin config) and starts the Eleventy watcher that does a full in-place build with proper context:

cloudron restart --app rmendes.net
# then confirm the build completed (see "Checking if the Eleventy Build Completed")

NEVER run eleventy directly via cloudron exec. It runs without the environment start.sh sets up (SITE_NAME, AUTHOR_NAME, plugin/secret config), producing a broken build that overwrites the in-place output with placeholder/default values. The ONLY safe full-rebuild trigger is cloudron restart. (The old "manual release swap" recipe here is obsolete — the swap is disabled and the manual build was unsafe.)

Rollback After a Bad Deploy

Because deploys reuse one in-place release dir (no per-deploy timestamped releases, atomic swap disabled), there is no symlink to repoint for rollback. To roll back, redeploy the previous image: check out the prior code/submodule state and make deploy SITE=<site> APP=<app>, then confirm the build completed. (A bad content/data state is instead fixed forward + cloudron restart.)

Workspace Context

This repo is part of the Indiekit development workspace at /home/rick/code/indiekit-dev/. See the workspace CLAUDE.md for the full repository map and relationships between all repos.

Related Repositories (all under /home/rick/code/indiekit-dev/)

  • indiekit/ - Upstream Indiekit fork (Lerna monorepo)
  • indiekit-eleventy-theme/ - Eleventy theme (this repo's submodule)
  • indiekit-endpoint-*/ - Custom endpoint plugins (@rmdes/*)
  • indiekit-post-type-page/ - Page post type plugin
  • indiekit-syndicator-*/ - Custom syndicator plugins (@rmdes/*)

References

  • Cloudron packaging guide: /home/rick/code/cloudron-skills/packaging-cloudron-apps/SKILL.md
  • Indiekit lessons learned: /home/rick/code/cloudron-skills/packaging-cloudron-apps/indiekit-lessons-learned.md
  • taiga-app (symlink pattern): https://git.cloudron.io/cloudron/taiga-app