redBus — Distributed Bus-Booking Platform

A production-grade, event-driven, microservices implementation of the redBus design corpus. 17 Go microservices · Kafka event bus · Postgres + Redis + Elasticsearch + MinIO · Prometheus/Grafana/Loki/Jaeger · One command to boot the entire stack.

This repository is a faithful, runnable implementation of the architecture documents under design/architecture/. It is built to be easy to read, easy to run locally, and faithful to the patterns you would actually deploy in production: transactional outbox, idempotent producers/consumers, atomic seat locking, saga-orchestrated bookings, JWT auth with refresh tokens, gateway rate-limiting, and full observability.

---

Table of Contents

1. Quick Start
2. What you get out of the box
3. System Overview
4. The 17 Services
5. The Booking Saga, end-to-end
6. How data flows: Kafka events
7. Core engineering patterns
8. Storage layout
9. Project layout
10. Shared pkg/ library
11. Configuration & environment
12. Local development workflow
13. Testing
14. Observability
15. API quick reference
16. Production considerations
17. Glossary & FAQ

---

1. Quick Start

The entire platform — 17 microservices, Postgres, Redis, Kafka, Elasticsearch, MinIO, Prometheus, Grafana, Loki, Jaeger, plus a seed job that runs an end-to-end booking — boots with one command:

docker compose up --build

That's it. Wait for the seed container to print ✓ booking CONFIRMED and ==> Seed complete, then point your browser at:

URL	What it is
http://localhost:8080/healthz	Gateway health check
http://localhost:8080/v1/health/all	Aggregate health of every backend
http://localhost:3000	Grafana (anonymous admin enabled)
http://localhost:9090	Prometheus
http://localhost:16686	Jaeger traces
http://localhost:9001	MinIO console (minioadmin / minioadmin)
http://localhost:9200	Elasticsearch

To shut everything down and wipe volumes:

make down       # docker compose down -v

Prerequisites: Docker Desktop (or any Docker engine with Compose v2), ~6 GB free RAM, ports 3000/3100/4317/4318/5432/6379/8080-8096/9000/9001/9090/9092/9200/16686/29092 free on localhost.

---

2. What you get out of the box

When docker compose up finishes, you have:

• 17 independent Go services each running on its own HTTP port, each with its own Postgres database (*_db) and its own migrations applied automatically on startup.
• A pre-seeded happy-path booking: operator → bus → route → trip → seat inventory → user signup → search → quote → initiate booking → pay → confirmation, all driven by scripts/seed.sh executed by the seed container.
• A single entry point at http://localhost:8080 (gateway-svc) that reverse-proxies every /v1/* path to the right backend, performs JWT validation hints, and applies token-bucket rate limiting on hot paths.
• Full observability: every service exposes /healthz, /readyz, /version, /metrics; Prometheus scrapes; Grafana auto-provisions an overview dashboard; Loki stores logs; Jaeger receives traces (OTLP gRPC 4317 / HTTP 4318).
• Event bus pre-wired: every producer service starts an outbox relay that polls its own Postgres outbox table and publishes to Kafka with topics auto-created (booking.created, payment.success, notification.send, …).
• A REST collection at docs/api-collection.http that you can paste into the VS Code REST Client / HTTPie / Postman to exercise every flow by hand.

---

3. System Overview

                                ┌──────────────────────────────────────────┐
                                │            Client / Browser              │
                                └──────────────────────┬───────────────────┘
                                                       │ HTTPS / WSS
                                                       ▼
                                   ┌───────────────────────────────────┐
                                   │      gateway-svc  :8080           │  ← rate-limits, JWT hints,
                                   │ reverse proxy + /v1/health/all    │    request-ID injection
                                   └───┬───────────────────────────┬───┘
              ┌──────────┬─────────────┼───────────────────────────┼─────────────┬──────────────┐
              ▼          ▼             ▼                           ▼             ▼              ▼
        auth-svc    user-svc     operator-svc                trip-svc     inventory-svc    search-svc
         :8081      :8082          :8083                      :8084          :8085           :8086
                                                                                                ▲
                                                                                                │ ES index
              ┌───────────────────────────────────────────────────────────────────────┐         │
              ▼                                                                       │         │
        pricing-svc ◄── HTTP ──┐                                                      │         │
         :8087                 │                                                      │         │
                              booking-svc :8088  ── HTTP ──►  inventory-svc           │         │
                                  │ ▲                                                 │         │
                                  │ │   payment.success                               │         │
                                  ▼ │                                                 │         │
                              payment-svc :8089                                       │         │
                                                                                      │         │
        wallet-svc  notification-svc  recommendation-svc  review-svc  realtime-svc    │         │
         :8090         :8091             :8092               :8093        :8094       │         │
                                                                                      │         │
         admin-svc :8095 (read-only BFF)       analytics-svc :8096 (Kafka → Postgres) │         │
                                                                                      │         │
              ┌───────────────────────────────────────────────────────────────────┐   │         │
              │                              KAFKA  :9092                        │◄──┴─────────┘
              └───────────────────────────────────────────────────────────────────┘
                       ▲ outbox relay                ▲ outbox relay
                       │                             │
              ┌────────┴────────┐         ┌──────────┴──────────┐
              │   Postgres :5432 │         │     Redis :6379     │
              │  per-svc DBs     │         │ seat locks, JWT     │
              │  outbox, dedup   │         │ denylist, ratelimit │
              └──────────────────┘         └─────────────────────┘

There are three layers:

1. Infrastructure — Postgres 15, Redis 7, Kafka 7.6 (+ Zookeeper), Elasticsearch 8.13, MinIO, Prometheus, Grafana, Loki, Jaeger.
2. Microservices — 17 stateless Go services, each with its own database and event topics.
3. Bootstrap — a one-shot seed container that waits for everything to be healthy then runs an end-to-end booking against the gateway.

All services share one Go module workspace (go.work) and a single multi-stage Dockerfile builds all 17 binaries into one image. Compose then runs that image 17 times, each with a different ENTRYPOINT and a different port.

---

4. The 17 Services

Every service is structured the same way:

services/<svc>/
├── main.go              # wires deps via pkg/svcrun, starts HTTP server + Kafka consumers + outbox relay
├── internal/            # service.go (domain logic), http.go (routes), consumers.go (Kafka)
├── migrations/          # 0001_init.sql — applied on boot by pkg/migrator
├── go.mod / go.sum
└── Dockerfile           # thin wrapper around the shared multi-stage image

4.1 Service map

Service	Port	Purpose	Stores	Produces	Consumes
gateway-svc	8080	Single entry point. Reverse-proxies every /v1/* to the right backend. Token-bucket rate limits search & booking-initiate. Aggregates health. Injects X-Request-ID.	Redis	—	—
auth-svc	8081	Signup/login, password hashing, phone OTP, JWT access/refresh issue+refresh, refresh-token denylist.	Postgres, Redis	user.signed_up, user.logged_in	—
user-svc	8082	User profile, saved passengers, addresses, preferences.	Postgres	—	user.signed_up
operator-svc	8083	Bus operators and their fleet (buses, amenities).	Postgres	—	—
trip-svc	8084	Routes + scheduled trips. Emits trip.published so search-svc can index.	Postgres	trip events	—
inventory-svc	8085	The authoritative seat-state machine (AVAILABLE → LOCKED → BOOKED). Implements Redis-Lua atomic multi-seat lock + Postgres CAS write.	Postgres, Redis	seat.locked, seat.released, seat.booked	—
search-svc	8086	Elasticsearch indexer + query API. Indexes trips, serves GET /v1/search, autocomplete.	Elasticsearch	—	trip events
pricing-svc	8087	Quote calculator (dynamic pricing inputs → total). 15-minute TTL quotes referenced by booking.	Postgres	price.quote.created	—
booking-svc	8088	The saga orchestrator. Owns the booking lifecycle (INITIATED → SEATS_LOCKED → PAYMENT_PENDING → CONFIRMED/CANCELLED/EXPIRED/FAILED). Talks HTTP to pricing/inventory/payment, writes booking row + outbox event in one TX.	Postgres	booking.created, booking.seats_locked, booking.confirmed, booking.cancelled, booking.expired, notification.send	payment.success, payment.failed
payment-svc	8089	Idempotent fake PSP. Intent → process (success/fail) → refund.	Postgres	payment.initiated, payment.success, payment.failed, refund.initiated	—
wallet-svc	8090	Append-only ledger of money in/out, balance views.	Postgres	wallet.updated	refund events
notification-svc	8091	Channel deliveries (email/SMS/push — mock implementations).	Postgres	notification.delivered	notification.send
recommendation-svc	8092	Records user interactions, recomputes boosted scores per user.	Postgres	—	booking.confirmed, review.created
review-svc	8093	Post-trip reviews & ratings.	Postgres	review.created	booking.confirmed (to prompt)
realtime-svc	8094	WebSocket gateway with Redis pub/sub fanout for bus location / ETA streams.	Redis	—	bus.location.updated, eta.updated
admin-svc	8095	Read-only BFF that fans out to every backend over HTTP. Requires role=admin JWT.	Redis	—	—
analytics-svc	8096	Kafka → Postgres fact tables, daily KPI rollups.	Postgres	—	booking + payment + user events

4.2 Why this exact split?

• Operator / trip / inventory are split because their write rates and consistency models differ. Inventory needs atomic seat locking at peak; trip metadata is read-heavy and almost never changes.
• Pricing is a side-effect-free quote service so it can be cached and scaled horizontally without coordination.
• Booking is the only service that owns the saga; every other service is either an HTTP backend it calls or a Kafka consumer it produces to. Saga state lives in booking_saga so a crashed orchestrator can resume from a reconciler loop.
• Search is a CQRS-style read model: writes happen elsewhere; search-svc consumes change events and rebuilds an ES index it owns.
• Notification / recommendation / review / realtime / analytics are pure event consumers — they don't sit on the request path and can fall behind without breaking bookings.

---

5. The Booking Saga, end-to-end

This is the heart of the system. A successful booking is six HTTP calls plus four event hops, all idempotent and crash-safe.

Client                Gateway              booking-svc           pricing-svc       inventory-svc        payment-svc         Kafka
  │                       │                     │                      │                  │                   │                │
  │ POST /v1/bookings/initiate                                                                                                  │
  │ Idempotency-Key: K    │                     │                                                                               │
  │──────────────────────▶│ proxy ─────────────▶│                                                                               │
  │                       │                     │ POST /v1/pricing/quote                                                        │
  │                       │                     │─────────────────────▶│                                                        │
  │                       │                     │◀── 200 quote ────────│                                                        │
  │                       │                     │ POST /v1/inventory/lock {trip, seats, ttl=600s}                                │
  │                       │                     │─────────────────────────────────────────▶│                                    │
  │                       │                     │ Redis Lua: atomic multi-seat SET-NX + aux set, single hash slot {trip:date}    │
  │                       │                     │ Postgres CAS: UPDATE … WHERE state='AVAILABLE'                                 │
  │                       │                     │◀── 200 {lock_token, expires_at} ─────────│                                    │
  │                       │                     │ TX: INSERT bookings (state=SEATS_LOCKED) + INSERT outbox(booking.created,      │
  │                       │                     │     booking.seats_locked) + INSERT idempotency_keys                            │
  │                       │                     │ (commit)                                                                       │
  │                       │                     │◀── outbox relay polls + publishes ─────────────────────────────────────────────┤
  │◀─── 201 {booking_id, pnr, expires_at} ──────│                                                                                │
  │                                                                                                                              │
  │ POST /v1/bookings/{id}/pay                                                                                                   │
  │──────────────────────▶│ proxy ─────────────▶│ POST /v1/payments/intents                                                      │
  │                       │                     │──────────────────────────────────────────────────────▶│                       │
  │                       │                     │                                                       │ TX: insert payment    │
  │                       │                     │                                                       │ + outbox(payment.init)│
  │                       │                     │◀── 200 {payment_id} ──────────────────────────────────│                       │
  │                       │                     │ TX: state=PAYMENT_PENDING + outbox(payment.initiated)                          │
  │◀── 200 {payment_id} ──│                     │                                                                                │
  │                                                                                                                              │
  │ POST /v1/payments/intents/{id}/process {success:true}                                                                        │
  │──────────────────────▶│ proxy ─────────────────────────────────────────────────────────▶│                                    │
  │                       │                     │                                            │ TX: state=SUCCEEDED               │
  │                       │                     │                                            │ + outbox(payment.success)        │
  │                       │                     │                                            │ (commit)                          │
  │                       │                     │                                            │── relay ──▶ payment.success ─────▶│
  │                       │                     │◀── consumer payment.success ──────────────────────────────────────────────────│
  │                       │                     │ idempotent: INSERT processed_events (consumer_group, event_id) ON CONFLICT     │
  │                       │                     │ POST /v1/inventory/confirm {lock_token, booking_id}                            │
  │                       │                     │──────────────────────────────────────────▶│ TX: state=BOOKED                   │
  │                       │                     │◀── 200 ───────────────────────────────────│                                    │
  │                       │                     │ TX: state=CONFIRMED + outbox(booking.confirmed, notification.send)              │
  │                       │                     │ (commit)                                                                       │
  │                       │                     │── relay ──▶ booking.confirmed ────────────────────────────────────────────────▶│
  │                       │                     │── relay ──▶ notification.send ────────────────────────────────────────────────▶│
  │                                                                                                                              │
  │ GET /v1/bookings/{id}  → 200 {state: "CONFIRMED"}                                                                            │

What happens on the failure paths:

• Seat already taken → inventory-svc returns seat_taken → booking-svc returns 409 synchronously. No saga state created.
• Payment fails → payment.failed event → booking-svc consumer → release the inventory lock → emit booking.cancelled.
• Client never pays → bookings whose expires_at (lock TTL) elapses are picked up by the reconciler (internal.RunReconciler) → state moves to EXPIRED → lock auto-released by Redis TTL.
• Orchestrator crashes mid-saga → booking_saga.current_step + next_attempt_at lets the reconciler pick up where the dead instance left off.

Everything is idempotent: re-sending the same Idempotency-Key returns the original 201 response with the same booking_id and pnr; replayed Kafka events hit the processed_events PK and are dropped.

---

6. How data flows: Kafka events

Every state-mutating service uses the transactional outbox pattern. In one Postgres transaction it writes the business row and a row to its own outbox table; a background relay (pkg/outbox.NewRelay) polls unpublished rows and produces them to Kafka. This means:

• If the relay crashes, nothing is lost — the row is still in outbox with published_at IS NULL.
• If Kafka is down, the relay backs off and retries.
• If a service crashes mid-flight, atomicity is preserved because the outbox write is part of the same TX as the state change.

6.1 Canonical event topics (see pkg/events/events.go)

user.signed_up                booking.created
user.logged_in                booking.seats_locked
user.deleted                  booking.confirmed
                              booking.cancelled
payment.initiated             booking.expired
payment.success
payment.failed                seat.locked
refund.initiated              seat.released
refund.completed              seat.booked
                              seat.inventory.snapshot
notification.send
notification.delivered        bus.location.updated
                              eta.updated
review.created                coupon.applied
price.quote.created           wallet.updated
audit.events

6.2 The standard event envelope

Every event is wrapped in events.Envelope:

{
  "event_id":       "uuid",
  "event_type":     "booking.confirmed",
  "aggregate_type": "booking",
  "aggregate_id":   "uuid",
  "schema_version": 1,
  "occurred_at":    "2026-05-14T10:00:00Z",
  "producer":       "booking-svc",
  "trace_id":       "...",
  "idempotency_key":"K",
  "payload":        { ... },
  "headers":        { ... }
}

event_id + consumer_group give every consumer a primary key for replay protection (processed_events table).

6.3 Topic auto-creation

Each service that produces events calls kafka.EnsureTopics(...) in main.go with its expected topics, partitions, and replication factor. Local Kafka is configured with KAFKA_AUTO_CREATE_TOPICS_ENABLE=true as a belt-and-suspenders so consumers also create missing topics on first poll.

---

7. Core engineering patterns

These show up across every service. Read these once and the codebase will read fluently.

7.1 Atomic multi-seat lock — pkg/seatlock

Locking N seats for one trip is a single Redis call. The Lua script:

1. Checks every seat key (seatlock:{trip:date}:<seat>) — if any exists, returns TAKEN:<seatNo>.
2. Otherwise SET key token EX ttl NX for every seat and SADD to a per-trip aux set so bulk release is O(1).
3. Returns OK along with the generated UUID token the caller must present to release/confirm.

The hash tag {trip:date} is critical: it forces all seats for that trip+date onto the same Redis hash slot, which is the only way EVAL can touch multiple keys in Redis Cluster.

Layered on top: inventory-svc also performs a Postgres CAS UPDATE … WHERE state='AVAILABLE' so the durable store is the final source of truth and survives Redis failures.

7.2 Transactional outbox — pkg/outbox

Schema (shipped in every producing service's migration):

CREATE TABLE outbox (
    id BIGSERIAL PRIMARY KEY,
    aggregate_type TEXT NOT NULL,
    aggregate_id   TEXT NOT NULL,
    event_type     TEXT NOT NULL,
    topic          TEXT NOT NULL,
    partition_key  TEXT NOT NULL,
    payload        JSONB NOT NULL,
    headers        JSONB NOT NULL DEFAULT '{}'::jsonb,
    created_at     TIMESTAMPTZ NOT NULL DEFAULT now(),
    published_at   TIMESTAMPTZ
);

In domain code:

tx, _ := pool.BeginTx(ctx, ...)
// 1. write business row(s)
// 2. outbox.Enqueue(ctx, tx, outbox.Event{...})
tx.Commit(ctx)

In main.go:

relay := outbox.NewRelay(d.PG, d.Producer, d.Log)
go relay.Run(ctx)

7.3 Idempotency

Three places, three patterns:

1. HTTP requests (POST /v1/bookings/initiate): require an Idempotency-Key header; booking-svc stores (user_id, key) → response in idempotency_keys for 24h. Replays return the original 201.
2. Event producers: the outbox row carries a deterministic event_id that consumers can dedupe by.
3. Event consumers: every consumer inserts (consumer_group, event_id) into processed_events with ON CONFLICT DO NOTHING before applying side effects.

7.4 JWT auth — pkg/jwt, pkg/middleware

• Access tokens (default 60 min) + refresh tokens (default 30 days), HS256, separate audience/issuer claims.
• middleware.JWTAuth(iss) enforces; middleware.OptionalJWTAuth(iss) populates context but does not 401.
• middleware.RequireRole("admin") for admin-svc.
• Refresh-token revocation is a Redis denylist (auth:denylist:<jti> with TTL ≤ refresh TTL).

7.5 Token-bucket rate limiting — pkg/ratelimit

Single Lua script per request: HMGET key t,ts → refill → check → HSET key t,ts → return {allowed, retry_after_ms}. Applied at the gateway on:

• GET /v1/search: RATE_SEARCH_PER_SEC (default 60/s/IP).
• POST /v1/bookings/initiate: RATE_BOOKING_PER_MIN (default 10/min/user — keyed off JWT sub).

7.6 Hexagonal layout per service

internal/
  service.go     ← pure domain logic, takes pgxpool + redis client + logger
  http.go        ← chi routes that translate HTTP ↔ service calls
  consumers.go   ← Kafka consumer handlers, also call into service.go

No service imports another service's internal/. All cross-service comms is HTTP (synchronous) or Kafka (asynchronous).

7.7 Reverse-proxy gateway — services/gateway-svc/internal/proxy.go

Uses httputil.ReverseProxy with a custom Director that injects X-Request-ID and X-Forwarded-For, and an ErrorHandler that logs proxy_error and returns 502. Backend URLs are env-vars (AUTH_URL, BOOKING_URL, …).

7.8 Graceful lifecycle — pkg/svcrun

Every main.go reads as a wiring file:

ctx, cancel := svcrun.SignalContext()           // SIGINT/SIGTERM → cancel
defer cancel()

d := svcrun.Init(ctx, svcrun.Options{
    Service:          "booking-svc",
    DefaultHTTPAddr:  ":8088",
    UsePG:            true,
    UseKafkaProducer: true,
})

migrator.Run(ctx, d.PG, config.String("MIGRATIONS_DIR", "/app/migrations"))
// build service, mount routes, ensure topics, start relay, start consumers
svcrun.RunGroup(ctx, srv.Run, relay.Run, consumer1.Run, consumer2.Run)

RunGroup is the supervisor: if any goroutine errors, it cancels the context and waits for the rest to drain.

---

8. Storage layout

8.1 Postgres

A single Postgres 15 instance hosts one database per service. The bootstrap script infra/postgres/init.sql creates them on first boot:

auth_db        operator_db    pricing_db     wallet_db          recommendation_db
user_db        trip_db        booking_db     notification_db    analytics_db
               inventory_db                  review_db
                              payment_db

Each service runs SQL migrations from services/<svc>/migrations/0001_init.sql on startup via pkg/migrator. The schemas all follow the same shape: business tables, optional outbox, optional processed_events, optional idempotency_keys.

8.2 Redis

Use	Key shape
Seat locks	seatlock:{trip:date}:<seatNo> (Lua atomic)
Seat lock aux set	seatlock:trip:{trip:date}
Refresh-token denylist	auth:denylist:<jti>
Rate-limit buckets	ratelimit:<scope>:<id>
Realtime pub/sub	realtime:trip:<id>
OTP codes	otp:<phone>

8.3 Elasticsearch

One index (trips) maintained by search-svc from trip/inventory events. Query is GET /v1/search?src=<city>&dst=<city>&date=YYYY-MM-DD.

8.4 MinIO

S3-compatible object store provided for future use (PDF tickets, exports, etc.). Not required by the happy path.

8.5 Kafka

Single broker locally (replication 1, 6 partitions per business topic). Topics created lazily by services in main.go.

---

9. Project layout

redbus/
├── go.work                       # workspace: pkg + 17 service modules
├── go.work.sum
├── Makefile                      # dev / build / test / seed / e2e
├── Dockerfile                    # multi-stage, builds all 17 binaries into one image
├── Dockerfile.builder            # optional builder image
├── docker-compose.yml            # full local platform (infra + 17 services + seed)
├── README.md                     # this file
│
├── pkg/                          # shared Go library (one module)
│   ├── config        # env-var helpers (String/Int/Bool/Duration)
│   ├── errors        # AppError + codes (CodeUnauthorized, CodeInvalidInput, …)
│   ├── events        # topic constants + Envelope + payload structs
│   ├── grpcserver    # gRPC server scaffolding (reserved for future)
│   ├── httpserver    # chi server, JSON helpers, /healthz /readyz /metrics
│   ├── id            # UUID + PNR helpers
│   ├── idem          # idempotency key store
│   ├── jwt           # HS256 access/refresh issuer + verifier
│   ├── kafka         # consumer/producer wrappers + EnsureTopics
│   ├── logger        # log/slog setup
│   ├── metrics       # Prometheus registry + common metrics
│   ├── middleware    # JWTAuth, OptionalJWTAuth, RequireRole, RequestID, recoverer
│   ├── migrator      # SQL migration runner
│   ├── outbox        # transactional outbox + relay
│   ├── pglib         # pgxpool wiring + retry/backoff
│   ├── ratelimit     # Redis-Lua token bucket
│   ├── redislib      # go-redis client wiring
│   ├── seatlock      # Redis-Lua atomic multi-seat lock
│   ├── svcrun        # Init + SignalContext + RunGroup supervisor
│   ├── tracing       # OTLP exporter setup
│   └── validation    # request struct validation helpers
│
├── services/                     # 17 microservices, each a separate go module
│   ├── auth-svc/        :8081
│   ├── user-svc/        :8082
│   ├── operator-svc/    :8083
│   ├── trip-svc/        :8084
│   ├── inventory-svc/   :8085
│   ├── search-svc/      :8086
│   ├── pricing-svc/     :8087
│   ├── booking-svc/     :8088
│   ├── payment-svc/     :8089
│   ├── wallet-svc/      :8090
│   ├── notification-svc/:8091
│   ├── recommendation-svc/:8092
│   ├── review-svc/      :8093
│   ├── realtime-svc/    :8094
│   ├── admin-svc/       :8095
│   ├── analytics-svc/   :8096
│   └── gateway-svc/     :8080
│
├── infra/
│   ├── postgres/init.sql              # creates per-service databases
│   ├── prometheus/prometheus.yml      # scrape config
│   ├── grafana/provisioning/          # datasources + dashboards
│   └── loki/loki-config.yaml
│
├── scripts/
│   ├── wait-for-services.sh           # polls /healthz of every service
│   └── seed.sh                        # end-to-end booking against the running stack
│
├── tests/e2e/e2e.sh                   # end-to-end smoke test with assertions
│
├── docs/
│   ├── architecture.md                # runtime mermaid diagrams
│   └── api-collection.http            # REST Client / HTTPie collection
│
└── design/architecture/               # 30 deep-dive design docs (preserved)
    ├── 01-high-level-architecture.md
    ├── 06-seat-locking-system.md
    ├── 07-booking-engine.md
    ├── 19-api-gateway.md
    └── …

---

10. Shared pkg/ library

Everything generic lives in pkg/. Services import it via github.com/redbus/pkg/<sub>. Some of the most useful pieces:

Package	What it gives you
pkg/svcrun	Init(opts) returns deps (PG pool, Redis, Kafka producer, logger, base config). SignalContext() ties cancellation to SIGINT/SIGTERM. RunGroup(ctx, fn...) supervises N goroutines.
pkg/httpserver	New(name, addr, log) returns a *Server with a chi router pre-mounted with /healthz, /readyz, /version, /metrics. ReadJSON, WriteJSON, WriteError.
pkg/middleware	RequestID, Recoverer, JWTAuth, OptionalJWTAuth, RequireRole, UserIDFromCtx, RequestIDFromCtx.
pkg/jwt	NewIssuer(secret, iss, aud, accessTTL, refreshTTL), IssueAccess, IssueRefresh, Verify.
pkg/kafka	EnsureTopics, NewProducer, NewConsumer(brokers, topic, group, handler, log).Run(ctx).
pkg/outbox	Enqueue(ctx, tx, evt) inside a domain TX, NewRelay(pool, producer, log).Run(ctx) in main.
pkg/seatlock	Manager.Lock, Release, Confirm — Redis-Lua atomic ops.
pkg/ratelimit	Limiter.Allow(ctx, scope, id, capacity, refillPerSec) — token bucket via Lua.
pkg/events	Topic constants + Envelope + payload structs (BookingConfirmed, PaymentSuccess, …).
pkg/errors	AppError{Code, Message, Cause} with codes used by httpserver.WriteError to map to HTTP statuses.
pkg/migrator	Run(ctx, pool, dir) reads *.sql and applies in order.
pkg/tracing	OTLP gRPC exporter wired to OTEL_EXPORTER_OTLP_ENDPOINT (defaults to Jaeger 4317).

---

11. Configuration & environment

Every service is configured by environment variables. Defaults are sensible for local Docker Compose. Common ones (set via the x-svc-env anchor in docker-compose.yml):

Variable	Default	Notes
ENV	dev	Used for logging context.
REGION	local
LOG_LEVEL	info	debug for verbose.
SERVICE_NAME	per-svc	Used in logs/metrics/traces.
HTTP_ADDR	per-svc :80xx
POSTGRES_DSN	per-svc …/<svc>_db
REDIS_URL	redis://redis:6379/0
KAFKA_BROKERS	kafka:9092	Inside the compose network. Use localhost:29092 from your host machine.
JWT_SECRET	redbus-dev-secret-please-rotate	Rotate before exposing anywhere.
JWT_ISSUER	redbus.auth
JWT_AUDIENCE	redbus.api
JWT_ACCESS_TTL	60m
JWT_REFRESH_TTL	720h
MIGRATIONS_DIR	/app/migrations/<svc>

Service-specific:

• booking-svc: PRICING_URL, INVENTORY_URL, PAYMENT_URL.
• payment-svc: FAKE_PAYMENT_FAILURE_RATE (set 0.2 to simulate 20% failures).
• gateway-svc / admin-svc: one *_URL env var per downstream service.
• gateway-svc: RATE_SEARCH_PER_SEC (default 60), RATE_BOOKING_PER_MIN (default 10).
• search-svc: ELASTICSEARCH_URL.

---

12. Local development workflow

12.1 Make targets

make help      # list all targets
make dev       # docker compose up --build (foreground)
make up        # docker compose up -d --build (detached)
make down      # docker compose down -v (wipes volumes)
make logs      # tail logs for every service
make ps        # container status
make build     # go build every service (no Docker)
make tidy      # go mod tidy every service
make test      # go test -race ./... in pkg + every service
make seed      # run scripts/seed.sh against the running stack
make e2e       # run tests/e2e/e2e.sh with assertions
make clean     # tear down + prune dangling images

12.2 Working on one service

You can rebuild and restart just one container:

docker compose up --build -d booking-svc
docker compose logs -f booking-svc

Or run the binary directly outside Docker against the compose Postgres/Redis/Kafka:

cd services/booking-svc
POSTGRES_DSN="postgres://redbus:redbus@localhost:5432/booking_db?sslmode=disable" \
KAFKA_BROKERS="localhost:29092" \
REDIS_URL="redis://localhost:6379/0" \
PRICING_URL="http://localhost:8087/v1/pricing/quote" \
INVENTORY_URL="http://localhost:8085/v1/inventory" \
PAYMENT_URL="http://localhost:8089/v1/payments/intents" \
MIGRATIONS_DIR="./migrations" \
HTTP_ADDR=":8088" \
go run .

12.3 Adding a new service

1. Copy services/auth-svc/ to services/new-svc/, update go.mod module name.
2. Replace internal/service.go, internal/http.go, migrations/0001_init.sql.
3. Add the path to go.work.
4. Add a COPY services/new-svc/... line to the root Dockerfile.
5. Add an entry to docker-compose.yml using <<: *go-svc and a unique port + ENTRYPOINT.
6. (Optional) Add a /v1/new mount in gateway-svc/internal/http.go.

---

13. Testing

• Unit tests live next to the code they test (_test.go). Run with make test (uses -race).
• End-to-end smoke test is tests/e2e/e2e.sh. It performs:
  • health-check the gateway
  • create operator → bus → route → trip → seat inventory
  • signup → JWT
  • initiate → pay → process → poll for CONFIRMED
  • verify idempotency (Idempotency-Key replays produce the same booking)
  • print a pass/fail summary
• Seed script (scripts/seed.sh) is the happy-path drive; it is also what the seed compose container runs on every docker compose up.

To run the smoke test against an already-running stack:

make e2e

---

14. Observability

14.1 Health

Every service exposes:

• GET /healthz — liveness (process is up).
• GET /readyz — readiness (downstream deps reachable).
• GET /version — build info.
• GET /metrics — Prometheus exposition.

The gateway also exposes GET /v1/health/all which fans out to every backend and returns a single JSON aggregate. Handy for dashboards and CI smoke tests.

14.2 Metrics → Prometheus → Grafana

Prometheus (infra/prometheus/prometheus.yml) scrapes every service's /metrics every 15s. Grafana is pre-provisioned with:

• A Prometheus datasource pointing at http://prometheus:9090.
• A Loki datasource pointing at http://loki:3100.
• An overview dashboard under infra/grafana/dashboards/.

Open http://localhost:3000 — anonymous admin is enabled, so no login required for local development.

14.3 Logs → Loki

Services log via log/slog to stdout in JSON. Compose ships logs to Loki via the standard Docker log driver, queryable in Grafana under Explore.

14.4 Traces → Jaeger

If a service has OTEL_EXPORTER_OTLP_ENDPOINT set (default for tracing-enabled services), spans are emitted via OTLP gRPC to Jaeger (jaeger:4317). UI at http://localhost:16686.

---

15. API quick reference

The full collection is in docs/api-collection.http. Highlights:

Method	Path	What it does	Auth
POST	/v1/auth/signup	Create user, return access + refresh tokens.	—
POST	/v1/auth/login	Email/password → tokens.	—
POST	/v1/auth/otp/send	Send OTP to phone.	—
POST	/v1/auth/otp/verify	Verify OTP → tokens.	—
POST	/v1/auth/refresh	Rotate access token using a refresh token.	—
GET	/v1/users/me/profile	Current user profile.	Bearer
POST	/v1/operators	Create operator.	— (admin in prod)
POST	/v1/operators/{id}/buses	Add a bus.	—
POST	/v1/routes	Create a route.	—
POST	/v1/trips	Schedule a trip.	—
POST	/v1/inventory/trips/{id}/seats	Seed seats for a trip+date.	—
GET	/v1/search?src=&dst=&date=	Search trips.	—
POST	/v1/pricing/quote	Get a 15-min quote.	Bearer
POST	/v1/bookings/initiate	Start a booking (requires Idempotency-Key).	Bearer
GET	/v1/bookings/{id}	Get a booking.	Bearer
POST	/v1/bookings/{id}/pay	Create payment intent.	Bearer
POST	/v1/bookings/{id}/cancel	Cancel a booking.	Bearer
POST	/v1/payments/intents/{id}/process	(Fake PSP) succeed/fail an intent.	—
GET	/v1/wallets/me	Wallet balance + ledger.	Bearer
WS	/v1/realtime/trip/{id}	Live bus location + ETA stream.	Bearer
GET	/v1/admin/*	Read-only admin BFF.	Bearer (role=admin)
GET	/v1/analytics/*	KPI rollups.	Bearer (role=admin)

Error responses follow a single shape:

{ "error": { "code": "seat_taken", "message": "seat 12 is already locked" } }

---

16. Production considerations

This repo runs everything on one host for clarity. What changes when you go to production:

• Postgres: one database per service is preserved; in production each service gets its own primary + replicas. Schemas already encode the consistency contracts (CAS, FKs, idempotency tables).
• Redis: switch to a cluster; the {trip:date} hash tag in seatlock keys is already cluster-safe.
• Kafka: bump partition counts (we use 6 locally) and replication factor (1 locally → 3 in prod). Set producer acks=all, enable idempotent producers (the relay is idempotent at the application layer too).
• JWT secret: JWT_SECRET must be rotated and pulled from a secret manager. Refresh-token denylist is already in Redis.
• Rate limits: locally we apply at the gateway; in production, also enforce at the edge (CDN/WAF).
• Observability: replace anonymous Grafana with SSO; add alert rules; export traces to a multi-tenant backend.
• Disaster recovery: see design/architecture/17-disaster-recovery.md — the patterns (outbox, CDC, region pinning) are already shaped to support it.
• Schema evolution: every event carries schema_version; consumers must handle multiple versions during rollouts. See design/architecture/28-event-schema.md.

---

17. Glossary & FAQ

PNR — Passenger Name Record. Human-friendly booking identifier (bookings.pnr).

Saga — A long-running business transaction split into a sequence of local transactions, each emitting events that trigger the next. Compensation events run on failure. Owned here by booking-svc.

Outbox — A table written in the same Postgres transaction as the business row, then asynchronously published to Kafka by a relay. Guarantees at-least-once delivery with no dual-write race.

CAS — Compare-and-Swap. The UPDATE … WHERE state='AVAILABLE' pattern in trip_inventory — only succeeds if the seat is still available.

BFF — Backend-for-Frontend. admin-svc is a read-only BFF that aggregates per-page data so the admin UI does one call instead of N.

Idempotency key — A client-supplied string that lets the server safely deduplicate retried requests. Required on booking initiate.

Why not gRPC between services? — HTTP/JSON is fine for the scales in this repo and dramatically easier to debug locally with curl/Postman. pkg/grpcserver is reserved for future use.

Why not a service mesh? — Out of scope for this repo. In production this slots into an Istio/Linkerd mesh without code changes — health and /metrics endpoints are already in place.

Why does every service have its own outbox table? — Because outbox is bound to the producing aggregate's transaction; sharing it across services would require distributed transactions, which is exactly what outbox is designed to avoid.

How do I run only the infra (without the services)?

docker compose up -d postgres redis kafka zookeeper elasticsearch minio prometheus grafana loki jaeger

Then run services individually with go run . as described in §12.2.

Where do the 30 design docs come from? — They live under design/architecture/ and are the source of truth for the architecture this code implements. Topics include: high-level architecture, the seat-locking system, the booking engine, the payment system, the search system, the cache strategy, security, observability, deployment, Kubernetes design, CI/CD, disaster recovery, cost optimization, the API gateway, rate limiting, the auth system, AI recommendation, the admin system, the analytics pipeline, testing strategy, monorepo structure, DDD, event schema, and a machine-coding-round walkthrough.

---

Tech stack. Go 1.22 · chi v5 · pgx v5 · go-redis v9 · segmentio/kafka-go · golang-jwt v5 · gorilla/websocket · prometheus/client_golang · log/slog · OpenTelemetry · Postgres 15 · Redis 7 · Kafka 7.6 · Elasticsearch 8.13 · MinIO · Prometheus · Grafana 10 · Loki · Jaeger · Alpine 3.19.

License. See repository root.

Have fun. The fastest way to learn the system is to make dev, wait for ✓ booking CONFIRMED, then open docs/api-collection.http and break things.