feat(examples): add weaviate-live-rag continuously-fresh RAG demo

[jevy]

What this adds

A self-contained, git clone && docker compose up example — examples/weaviate-live-rag/ — that demonstrates continuously-fresh RAG: a support chatbot answers from a Weaviate index of help-center docs, and the moment a row changes in Postgres, TypeStream re-embeds it and the bot's answer changes seconds later. No reindex job, no consumer code, no Kafka to operate.

Built as the Tier-A "live ingestion" demo asset for devrel outreach. Data is fictional ("Northwind").

Architecture

Postgres(help_articles) --Debezium CDC--> Kafka --> TypeStream pipeline
   (kafkaSource unwrapCdc -> embeddingGenerator -> kafkaSink: help_article_embeddings)
   --> Weaviate sink connector --> Weaviate(HelpArticle, vectorizer:none)
                                          ^ nearVector
   chatbot (Node+Express) ----------------/   embed question -> top-3 -> grounded answer

The chatbot queries Weaviate directly; TypeStream's only job is keeping the index fresh. Everything runs from published images (server, kafka-connect, ui) plus a small Node chatbot and a one-shot bootstrap container.

How it works

• Pipeline-as-code: pipeline/help-articles.typestream.json is applied at startup via PipelineService.ApplyPipeline (gRPC). Config-as-code doesn't auto-create the Weaviate connector, so the pipeline ends in a kafkaSink and the bootstrap container registers the Weaviate Kafka Connect sink directly (config mirrors ConnectionService).
• Ephemeral by design: no named volumes — docker compose down wipes state, up re-seeds Postgres and re-runs bootstrap for a clean baseline every time.
• Cold-start readiness gate: bootstrap only prints "Demo is live" once all docs are queryable in Weaviate (EXPECTED_DOCS), eliminating the query-before-ready window.
• TypeStream UI included (caddy + envoy + uiv2) at http://localhost:5173 — the applied pipeline shows up as a running job with live throughput.

Verified demo moves

Move	Action	Result
Baseline	ask "How long is the free trial?"	"14 days"
Baseline	ask "Do you support SSO / Okta?"	"I don't have that information" (grounded)
UPDATE	psql < demo/update-trial.sql	bot flips to "30 days"; Weaviate stays 10 objects (in-place upsert)
INSERT	psql < demo/insert-sso.sql	bot answers from the brand-new SSO doc

Built to retarget

The reusable core (Postgres, CDC, redpanda, server, bootstrap core, chatbot shell) is vendor-neutral. Only two files encode "Weaviate":

• bootstrap/target.sh — create_collection + register_sink_connector
• chatbot/retriever.js — retrieve(question, k) -> [{title, body}]

So cloning this to Qdrant/Neo4j/etc. is "swap the sink + the retriever."

Notes

• Requires OPENAI_API_KEY in .env (gitignored); used by the server to embed docs and the chatbot to embed queries (same model, text-embedding-3-small).
• Deletes are out of scope (insert + update only).

🤖 Generated with Claude Code