Technical reference for AI assistants modifying the thepopebot NPM package source code.
Architecture: Event Handler (Next.js) creates agent-job/* branches → launches Docker agent container locally (Claude Code, Pi, etc.) → task executed → PR created → auto-merge → notification. Agent jobs log to logs/{JOB_ID}/.
The npm package (api/, lib/, config/, bin/) is published to npm. In production:
- Event handler: Docker image bakes the npm package, Next.js app source (
web/), and.nextbuild output. User project directories (agent-job/,event-handler/,skills/,skills-library/,.env,data/, etc.) are individually volume-mounted into/app. The full project is also mounted at/projectfor git access. Runsserver.jsvia PM2 behind Traefik reverse proxy. lib/paths.js: ExportsPROJECT_ROOT(process.cwd()). This is how the installed npm package finds the volume-mounted user project files.- Agent-job containers: Ephemeral Docker containers clone
agent-job/*branches separately — use named volumes for workspace. Seedocker/CLAUDE.md. - Local install: Gives users CLI tools (
init,setup,upgrade) and configuration scaffolding.
All event handler logic, API routes, library code, and core functionality lives in the npm package (api/, lib/, config/, bin/). This is what users import when they import ... from 'thepopebot/...'.
The templates/ directory contains only files that get scaffolded into user projects via npx thepopebot init. Templates are for user-editable configuration and thin wiring — things users are expected to customize or override. Never add core logic to templates.
When adding or modifying event handler code, always put it in the package itself (e.g., api/, lib/), not in templates/. Templates should only contain:
- Configuration files users edit (
agent-job/SYSTEM.md,agent-job/CRONS.json,event-handler/TRIGGERS.json, etc.) - GitHub Actions workflows
- Docker compose (
docker-compose.yml) - CLAUDE.md files for AI assistant context in user projects
Next.js app source files (app/, next.config.mjs, server.js, etc.) live in web/ at the package root. These are built into the Docker image — NOT scaffolded to user projects.
Files in managed directories are auto-synced (created, updated, and deleted) by init to match the package templates exactly. Users should not edit these files — changes will be overwritten on upgrade. Managed paths are defined in bin/managed-paths.js:
.github/workflows/— CI/CD workflowsdocker-compose.yml,.dockerignore— Docker config.gitignore— Git ignore rules
/
├── api/ # GET/POST handlers for all /api/* routes
├── lib/
│ ├── actions.js # Shared action executor (agent, command, webhook) — threads scope/agent_backend/llm_model/user_id
│ ├── cron.js # Cron scheduler (loads CRONS.json) + reloadCrons() for hot-reload
│ ├── triggers.js # Webhook trigger middleware (loads TRIGGERS.json) + reloadTriggers()
│ ├── paths.js # Exports PROJECT_ROOT (process.cwd())
│ ├── git-commands.js # Single source of truth for workspace git commands (pull/commit/push/pull-push/create-pr) + prompts
│ ├── ai/ # LLM integration (chat stream, SDK adapters, helper LLM, session manager, scope, system prompt)
│ ├── auth/ # NextAuth config, requireAuth/requireAdmin helpers, middleware, server actions, components
│ ├── channels/ # Channel adapters (base class, Telegram, factory) + slash commands
│ ├── chat/ # Chat route handler, server actions, React UI components
│ ├── cluster/ # Worker clusters (roles, triggers, Docker containers)
│ ├── code/ # Code workspaces (server actions, terminal view, WebSocket proxy)
│ ├── containers/ # Container SSE streaming (status + log tail)
│ ├── db/ # SQLite via Drizzle (schema, migrations, api-keys, messages, oauth-tokens, user-channels)
│ ├── tools/ # Job creation, GitHub API, Telegram, Docker, AssemblyAI
│ ├── voice/ # Voice input (AssemblyAI streaming transcription)
│ ├── oauth/ # OAuth state encryption (helper.js) + provider presets (providers.js)
│ ├── github-api.js # Low-level GitHub REST helper used by tools/github.js + setup
│ ├── llm-providers.js # BUILTIN_PROVIDERS table (slug → baseUrl, defaults, anthropicEndpoint)
│ ├── maintenance.js # Hourly cleanup cron (expired agent-job API keys, etc.)
│ ├── config.js # getConfig/setConfig + CONFIG_KEYS allowlist + DEFAULTS
│ └── utils/
│ ├── render-md.js # Markdown {{include}} processor
│ └── random-name.js # Random container name generator
├── config/
│ ├── index.js # withThepopebot() Next.js config wrapper
│ └── instrumentation.js # Server startup hook (loads .env, starts crons)
├── bin/ # CLI entry point (init, setup, reset, diff, upgrade)
├── setup/ # Interactive setup wizard
├── web/ # Next.js app source (baked into Docker image, NOT scaffolded)
│ ├── app/ # Next.js app directory (pages, layouts, routes)
│ ├── server.js # Custom Next.js server with WebSocket proxy
│ ├── next.config.mjs # Next.js config wrapper
│ ├── instrumentation.js # Server startup hook
│ ├── middleware.js # Auth middleware
│ └── postcss.config.mjs # PostCSS/Tailwind config
├── templates/ # Scaffolded to user projects (see rule above)
├── docs/ # Extended documentation
└── package.json
Exports defined in package.json exports field. Pattern: thepopebot/{module} maps to source files in api/, lib/, config/. Includes ./cluster/*, ./voice/* exports. Add new exports there when creating new importable modules.
Settings/admin pages use shared components from lib/chat/components/settings-shared.jsx. See lib/chat/components/CLAUDE.md for the full UI standards (button tiers, dialogs, save feedback, delete confirmation, spacing, etc.). Follow these standards when adding new settings pages.
Run npm run build before publish. esbuild compiles lib/chat/components/**/*.jsx, lib/auth/components/**/*.jsx, lib/code/*.jsx, lib/cluster/components/**/*.jsx to ES modules.
SQLite via Drizzle ORM at data/thepopebot.sqlite (override with DATABASE_PATH). Auto-initialized on server start. See lib/db/CLAUDE.md for schema details, CRUD patterns, and column naming.
All schema changes MUST go through the migration workflow.
- NEVER write raw
CREATE TABLE,ALTER TABLE, or any DDL SQL manually - NEVER modify
initDatabase()to add schema changes - ALWAYS make schema changes by editing
lib/db/schema.jsthen runningnpm run db:generate
/api routes are for external callers only. They authenticate via x-api-key header or webhook secrets (Telegram, GitHub). Never add session/cookie auth to /api routes.
Browser UI uses fetch route handlers colocated with pages. All authenticated browser-to-server calls use Next.js route handlers (route.js files in web/app/) that check auth() session. Do NOT use server actions for data fetching — they cause page refresh issues. Handler implementations live in lib/chat/api.js; route files are thin re-exports.
/stream/* is for actual SSE streaming only. Four endpoints use Server-Sent Events: /stream/chat (AI SDK streaming), /stream/containers (Docker container status), /stream/containers/logs?name=<container> (live container log tail), /stream/cluster/[clusterId]/logs (cluster logs). All other fetch routes are colocated with their page directories.
| Caller | Mechanism | Auth | Location |
|---|---|---|---|
| External (cURL, GitHub Actions, Telegram) | /api route handler |
x-api-key or webhook secret |
api/index.js |
| Browser UI (data/mutations) | Fetch route handler colocated with page | auth() session check |
web/app/<page>/route.js |
| Browser UI (SSE streaming) | EventSource / AI SDK streaming | auth() session check |
web/app/stream/ |
Shared executor for cron jobs and webhook triggers (lib/actions.js). Three action types: agent (Docker LLM container), command (shell command), webhook (HTTP request). See lib/CLAUDE.md for detailed dispatch format, cron/trigger config, and template tokens.
See lib/ai/CLAUDE.md for the provider table and model defaults. Key: LLM_PROVIDER + LLM_MODEL env vars, LLM_MAX_TOKENS defaults to 4096.
- Code Workspaces: Interactive Docker containers with in-browser terminal. See
lib/code/CLAUDE.md. - Cluster Workspaces: Groups of Docker containers spawned from role definitions with triggers. See
lib/cluster/CLAUDE.md.
Both use lib/tools/docker.js for container lifecycle via Unix socket API.
Skill source files live in skills-library/<skill-name>/SKILL.md (canonical store). The skills/ directory is the activation surface — each entry there is a symlink to ../skills-library/<skill-name>, and only symlinked skills are visible to coding agents. Removing a symlink in skills/ deactivates the skill without deleting its source.
init populates skills/ with one symlink per bundled skill only on first install (when skills/ did not pre-exist). Subsequent init/upgrade runs leave skills/ untouched — new bundled skills land in skills-library/ un-activated, and the user must ln -s ../skills-library/<X> skills/<X> to activate.
Each skill has a SKILL.md with YAML frontmatter (name, description). The {{skills}} template variable in markdown files resolves skill descriptions at runtime by scanning skills/ (which follows symlinks via lib/utils/render-md.js). Default skills: agent-job-secrets, agent-job-dm, agent-job-background, playwright-cli. Agent-specific bridges (.claude/skills, .pi/skills, etc.) symlink to skills/.
User-authored skills live in skills-library/<your-skill>/, activated via symlink in skills/. Don't put real files in skills/.
Config markdown files support {{ filepath.md }} includes (resolved relative to project root) and built-in variables ({{datetime}}, {{skills}}), powered by lib/utils/render-md.js.
Runtime config lives in the settings DB (lib/db/settings), not .env. The event handler reads via getConfig(key) and writes via the admin UI or setup/lib/sync.mjs during initial setup. .env only carries bootstrap values that must exist before the DB is available (e.g. AUTH_SECRET, DATABASE_PATH).
Agent-job containers are launched locally by the event handler (runAgentJobContainer in lib/tools/docker.js). Their env vars are built at launch time from settings DB values via buildAgentAuthEnv() plus per-job parameters (LLM_MODEL, SCOPE, USER_ID, AGENT_JOB_TOKEN, APP_URL, etc.). USER_ID is also set on every other container that carries AGENT_JOB_TOKEN (interactive, headless, SDK adapter), so skills like agent-job-dm and agent-job-background resolve the originator consistently across chat and agent-job modes. Agent jobs do not consume GitHub repository variables at runtime.
setup/lib/sync.mjs may still write certain values to GitHub repo variables/secrets (for CI workflows like the npm publish pipeline), but those are separate from the runtime config path used by chat and agent jobs.