docs: add infrastructure route tier, API standardization roadmap, and updated docs

AGENTS.md:
- Add infrastructure route tier (healthz, readyz, metrics) — any domain, no auth
- Add /api/v1/docs to public (any domain) tier
- Renumber all tiers (1-6)

roadmap.md:
- Add "Standardized API Response Envelope & Pagination" — Critical
- Add "Request Trace IDs" — Critical
- Add "Structured Error Codes" — Critical
- Consolidate API pagination into response envelope item
- Document current response inconsistencies, proposed format, migration strategy

architecture.md, deployment.md, api-reference.md, README.md:
- Update docs to reflect base-domain auth, health endpoints, metrics, logging config

Author: ngoyal16

AGENTS.md

@@ -59,6 +59,7 @@ Migrations are versioned and tracked in `common.schema_migrations`.
 - **Middleware chain (users):** `Auth(jwt) → TenantResolver(subdomain/header + membership check) → Handler`
 - **Password hash** is tagged `json:"-"` on the User model — never exposed in API responses.
 - **Email enumeration** is prevented: login returns the same error for wrong email and wrong password.
+- **Base domain restriction:** When `AEGIS_BASE_DOMAIN` is set, all public auth endpoints (register, login, logout, forgot-password, reset-password, MFA validate, verify-email) are blocked on org subdomains via `baseOnlyMiddleware`. Auth flows must happen on the base domain only. Cookies are set with `Domain=.baseDomain` for cross-subdomain sharing. The UI redirects users from subdomain auth pages to the base domain with a `?return_to=` param.
 
 ### Agent Authentication
 
@@ -143,6 +144,13 @@ Each org tracks a `schema_version` in `common.organizations`. This allows:
 - **Context:** Use `middleware.UserFromContext(ctx)` and `middleware.OrgFromContext(ctx)` to access the authenticated user and current org.
 - **Tenant store:** Use `tenantStore(r)` helper in handlers (calls `middleware.TenantStoreFromContext`).
 - **Imports:** Group as stdlib → external → internal.
+- **Logging:** Use `log/slog` (Go stdlib). Never use `fmt.Printf`, `log.Printf`, or `log.Println` for application logging.
+  - `slog.Info("message", "key", value)` — normal operational events
+  - `slog.Warn("message", "key", value)` — degraded but recoverable situations
+  - `slog.Error("message", "error", err)` — failures that need attention
+  - `slog.Debug("message", "key", value)` — verbose tracing (only visible at debug level)
+  - Always include structured key-value pairs, not formatted strings
+  - For request-scoped logging, use `logger.FromContext(ctx)` (future OTel trace ID injection)
 
 ### React UI
 
@@ -214,10 +222,12 @@ ci: add Docker build verification to CI
 
 ### Route Tiers
 
-1. **Public** — no auth required: `/api/v1/auth/register`, `/api/v1/auth/login`, `/api/v1/auth/logout`
-2. **Authenticated** — JWT cookie required: `/api/v1/auth/me`, `/api/v1/orgs`, `/api/v1/config/features`
-3. **Protected** — JWT + org context (subdomain/X-Org-Slug + membership verified): findings, projects, members, tokens, dashboard
-4. **Agent** — Bearer token (org resolved from subdomain/header, no membership check): `/api/v1/agent/*`
+1. **Infrastructure (any domain, no auth)** — registered on the top-level mux outside the API server, bypasses all middleware. Accessible on base domain, org subdomains, custom domains, IPs — anywhere. Used by load balancers, Kubernetes probes, and monitoring systems: `/healthz`, `/readyz`, `/metrics`
+2. **Public (base domain only)** — no auth required, but blocked on org subdomains when `AEGIS_BASE_DOMAIN` is set: `/api/v1/auth/register`, `/api/v1/auth/login`, `/api/v1/auth/logout`, `/api/v1/auth/forgot-password`, `/api/v1/auth/reset-password`, `/api/v1/auth/mfa/validate`, `/api/v1/auth/verify-email`
+3. **Public (any domain)** — no auth, works everywhere: `/api/v1/config/auth`, `/api/v1/docs`, `/api/v1/docs/openapi.yaml`
+4. **Authenticated** — JWT cookie required: `/api/v1/auth/me`, `/api/v1/orgs`, `/api/v1/config/features`
+5. **Protected** — JWT + org context (subdomain/X-Org-Slug + membership verified): findings, projects, members, tokens, dashboard
+6. **Agent** — Bearer token (org resolved from subdomain/header, no membership check): `/api/v1/agent/*`
 
 ### Request/Response
 
@@ -335,6 +345,15 @@ Transactional emails (password reset, MFA codes, etc.) are sent via SMTP. Config
 
 **Production:** Set real SMTP credentials (SendGrid, AWS SES, Mailgun, etc.) in `.env`.
 
+### Logging
+
+| Variable | Default | Description |
+|---|---|---|
+| `LOG_LEVEL` | `info` | Minimum log level: `debug`, `info`, `warn`, `error` |
+| `LOG_FORMAT` | `text` | Output format: `text` (human-readable) or `json` (structured for log aggregation) |
+
+Uses Go's stdlib `log/slog`. JSON format is recommended for production with log aggregation (ELK, Loki, CloudWatch, etc.).
+
 ---
 
 ## Testing

README.md

@@ -47,6 +47,43 @@ npm run dev
 # Open http://localhost:5173
 ```
 
+### Subdomain Testing (local)
+
+Aegis supports subdomain-based org resolution (e.g., `acme.aegis.io`). To test this locally, we use [`lvh.me`](http://lvh.me) — a public domain where `*.lvh.me` resolves to `127.0.0.1`. No `/etc/hosts` changes needed.
+
+**Setup:**
+
+```bash
+# .env (already set by default in .env.example)
+AEGIS_BASE_DOMAIN=lvh.me
+
+# Restart
+docker compose up --build -d
+```
+
+**Usage:**
+
+| URL | What happens |
+|---|---|
+| `http://lvh.me:8080` | Base domain — no subdomain, header-only mode |
+| `http://test.lvh.me:8080` | Resolves org with slug `test` from subdomain |
+| `http://acme.lvh.me:8080` | Resolves org with slug `acme` from subdomain |
+| Any new org slug | Works instantly — `*.lvh.me` is a wildcard |
+
+**Testing the mismatch guard:**
+
+```bash
+# This works — subdomain matches
+curl -b cookies.txt http://test.lvh.me:8080/api/v1/findings
+
+# This is REJECTED (400) — subdomain says "test" but header says "acme"
+curl -b cookies.txt http://test.lvh.me:8080/api/v1/findings \
+  -H "X-Org-Slug: acme"
+# → {"error":"X-Org-Slug header conflicts with subdomain"}
+```
+
+> **Production:** Set `AEGIS_BASE_DOMAIN=aegis.io` (or your domain). Configure DNS with a wildcard `*.aegis.io → your-server-ip`.
+
 ---
 
 ## Architecture
@@ -120,6 +157,15 @@ aegis/
 │       │   └── dashboard.go    # Dashboard stats
 │       ├── auth/               # JWT + bcrypt service
 │       ├── config/             # Env-based configuration
+│       ├── email/
+│       │   ├── email.go        # SMTP transport service
+│       │   └── templates/      # HTML email templates
+│       │       ├── layout.go           # Shared base layout + helpers
+│       │       ├── login_alert.go      # New sign-in notification
+│       │       ├── password_reset.go   # Reset link email
+│       │       ├── password_changed.go # Change confirmation
+│       │       ├── verify_email.go     # Email verification link
+│       │       └── mfa_code.go         # MFA OTP code
 │       ├── middleware/
 │       │   ├── auth.go         # JWT cookie validation
 │       │   ├── tenant.go       # Org resolution + membership check
@@ -153,7 +199,7 @@ aegis/
 │       └── components/
 │           ├── app-sidebar.tsx  # Left nav + user menu
 │           ├── top-nav.tsx      # Breadcrumb header
-│           ├── org-project-switcher.tsx
+│           ├── org-switcher.tsx
 │           ├── metric-card.tsx  # Dashboard stat cards
 │           ├── severity-badge.tsx
 │           └── ui/             # shadcn/ui primitives
@@ -198,7 +244,7 @@ All configuration is via environment variables:
 | Method | Endpoint | Description |
 |---|---|---|
 | POST | `/api/v1/orgs` | Create org (creator = owner) |
-| GET | `/api/v1/orgs` | List user's orgs |
+| GET | `/api/v1/orgs` | List user's orgs + `base_domain` config |
 | GET | `/api/v1/orgs/{slug}` | Get org by slug |
 
 ### Feature Flags (authenticated)

docs/api-reference.md

@@ -617,6 +617,21 @@ Current flags: `signup`, `invite_only`, `scan_docker_mode`, `public_api`
 
 ---
 
+### GET `/config/auth`
+
+Returns auth configuration. **Public endpoint** — no authentication required. Used by the UI to detect subdomain mode before the user has a session.
+
+**Response (200):**
+```json
+{
+  "base_domain": "aegis.io"
+}
+```
+
+When `AEGIS_BASE_DOMAIN` is not set, `base_domain` is an empty string.
+
+---
+
 ## Scans 🔒🏢 (Read-Only)
 
 Requires org context (`X-Org-ID` or `X-Org-Slug` header).

docs/architecture.md

@@ -99,8 +99,10 @@ This is validated by regex `^org_[a-f0-9]{32}$` to prevent SQL injection.
    │  └─ Inject user into context
    │
 5. Tenant Middleware (for org-scoped routes):
-   │  ├─ Read subdomain from Host header (production)
-   │  ├─ Or read X-Org-ID / X-Org-Slug header (dev)
+   │  ├─ Try subdomain from Host header (production, if AEGIS_BASE_DOMAIN set)
+   │  ├─ Try custom domain lookup (if host ≠ base domain)
+   │  ├─ Reject if X-Org-* header conflicts with subdomain (400)
+   │  ├─ Fallback: X-Org-ID / X-Org-Slug header (dev mode)
    │  ├─ Load org from DB
    │  ├─ Verify user is a member of the org
    │  ├─ Create schema-scoped Store
@@ -130,9 +132,32 @@ This is validated by regex `^org_[a-f0-9]{32}$` to prevent SQL injection.
 
 ## Auth Flow
 
+### Base Domain Restriction
+
+When `AEGIS_BASE_DOMAIN` is set (e.g., `aegis.io`), all auth flows are restricted to the base domain only. Requests to org subdomains (e.g., `acme.aegis.io/api/v1/auth/login`) are rejected with a 403 error.
+
+This applies to: register, login, logout, forgot-password, reset-password, MFA validate, MFA send-email-otp, and verify-email endpoints.
+
+**Defense in depth:**
+- **Server-side:** `baseOnlyMiddleware` rejects auth API calls from any host that isn't the exact base domain (subdomains, IPs, unknown hostnames are all blocked)
+- **UI-side:** `useSubdomainAuthRedirect` hook redirects auth pages to the base domain
+- **Cookie scoping:** Auth cookies are set with `Domain=.aegis.io` so they work across all subdomains
+
+**Login redirect flow:**
+```
+User visits acme.aegis.io (no session)
+  → UI redirects to aegis.io/login?return_to=https://acme.aegis.io/
+  → User logs in on aegis.io (cookie set with Domain=.aegis.io)
+  → UI redirects back to acme.aegis.io/ (cookie is valid on subdomain)
+```
+
+The `return_to` parameter is validated to prevent open redirect attacks — only URLs sharing the same base domain are allowed.
+
+**Exceptions:** `GET /api/v1/auth/me` works on any subdomain (it's authenticated, not public, and the UI needs it to check session status).
+
 ### Registration
 ```
-POST /api/v1/auth/register
+POST /api/v1/auth/register  (base domain only when AEGIS_BASE_DOMAIN set)
   │
   ├─ Check feature flag: signup enabled?
   ├─ Validate email, password (8+ chars, upper+lower+digit), name
@@ -142,22 +167,23 @@ POST /api/v1/auth/register
   ├─ Create default org ("Name's Org")
   ├─ Add user as org owner
   ├─ Generate JWT (24h TTL)
-  └─ Set aegis_token HttpOnly cookie
+  └─ Set aegis_token HttpOnly cookie (Domain=.baseDomain when set)
 ```
 
 ### Login
 ```
-POST /api/v1/auth/login
+POST /api/v1/auth/login  (base domain only when AEGIS_BASE_DOMAIN set)
   │
   ├─ Find user by email
   ├─ Compare bcrypt hash (same error for wrong email/password)
+  ├─ If MFA enabled: return mfa_required + short-lived MFA token
   ├─ Generate JWT
-  └─ Set aegis_token HttpOnly cookie
+  └─ Set aegis_token HttpOnly cookie (Domain=.baseDomain when set)
 ```
 
 ### Session Check
 ```
-GET /api/v1/auth/me
+GET /api/v1/auth/me  (works on any subdomain)
   │
   ├─ Auth middleware validates cookie
   ├─ Load user + their orgs
@@ -190,5 +216,7 @@ Each org's data is fully isolated at the database level:
 | Auth (Users) | bcrypt (cost 12), JWT (HS256, golang-jwt/v5) |
 | Auth (Agents) | Bearer tokens, bcrypt-hashed, per-org schema |
 | Org Resolution | Subdomain (`acme.aegis.io`) or `X-Org-Slug` header |
+| Logging | `log/slog` (Go stdlib), configurable level/format via `LOG_LEVEL`/`LOG_FORMAT` |
+| Observability | OpenTelemetry metrics, Prometheus exporter (`/metrics`) |
 | Deployment | Docker, Docker Compose |
 | UI Embedding | Go `embed` package (SPA served from binary) |

docs/deployment.md

@@ -28,9 +28,16 @@ POSTGRES_DB=aegis
 AEGIS_PORT=8080
 AEGIS_ALLOWED_ORIGINS=https://your-domain.com
 
-# Subdomain-based org resolution (optional)
-# Set this to enable acme.aegis.io style org URLs
-# AEGIS_BASE_DOMAIN=aegis.io
+# Subdomain-based org resolution
+# Development: lvh.me (*.lvh.me resolves to 127.0.0.1)
+# Production: your-domain.com (configure wildcard DNS: *.your-domain.com)
+AEGIS_BASE_DOMAIN=lvh.me      # dev
+# AEGIS_BASE_DOMAIN=aegis.io  # production
+
+# Logging (default: info level, text format)
+# Use json format for production log aggregation (ELK, Loki, etc.)
+LOG_LEVEL=info
+LOG_FORMAT=text
 ```
 
 Generate a strong JWT secret: