-
Notifications
You must be signed in to change notification settings - Fork 14
feat(chat): add Mastra chat-api prototype and OpenRouter smoke #5126
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
rschlaefli
wants to merge
49
commits into
v3
Choose a base branch
from
codex/mastra-chat-openrouter-smoke
base: v3
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
49 commits
Select commit
Hold shift + click to select a range
ee4dbd5
docs: plan Klicker MCP server and chat integration (#5080)
rschlaefli b38895a
feat(mcp-student): add student practice MCP (#5090)
rschlaefli 2e3caa2
feat(apps/chat): LTI semi-anonymous guest access (#5083)
rschlaefli 079a691
Merge branch 'v3' of https://github.com/uzh-bf/klicker-uzh into v3-ai
TheHummel c0fc217
docs: add Mastra framework evaluation for the chat platform
rschlaefli fcb5be5
docs: add Mastra chat prototype plan (Scope A+ evaluation harness)
rschlaefli 23823bf
chore(prototype): add S0.5 measurement queries + execution runbook
rschlaefli 0718c45
docs(prototype): add build-session kickoff runbook
rschlaefli 6f96c25
feat(prototype): scaffold S0 engine project (env, db, harness)
rschlaefli 003f971
feat(prototype): S0 engine spine + S0.5 measurement + synthetic fixture
rschlaefli 36272bc
feat(prototype): S1 retrieval + guardrails (code; live validation pen…
rschlaefli 27ee1b2
feat(prototype): S3/S4/S5 DIY branch-correct memory (offline-proven)
rschlaefli 91ea6af
feat(prototype): S2 DB-backed skill source (offline-proven) + model-t…
rschlaefli d96e2a1
feat(prototype): S7 eval dataset (model-independent fixture)
rschlaefli b6d0aa3
fix(prototype): use Chat Completions API (S1 live-validated)
rschlaefli 65ce15b
feat(prototype): S3 update_profile tool + context injection (live-val…
rschlaefli 098a1e2
feat(prototype): S4 embedding ranking on branch-restricted candidates…
rschlaefli 82548c7
feat(prototype): S6 two-level supervisor + DB roster (live-validated)
rschlaefli ef7d71f
feat(prototype): S7 eval runner over course-question dataset (live)
rschlaefli 798f276
feat(prototype): S2 skills applied live (progressive disclosure)
rschlaefli 64b8022
feat(prototype): S5 live summarize + measured token savings
rschlaefli 88e24ad
docs(prototype): backfill S0.5-S7 verdicts + final matrix and go/no-go
rschlaefli 26a4ef0
docs(prototype): record S0.5 measurement numbers (fixture-only)
rschlaefli f3e7f3b
merge: #5118 Mastra evaluation report into prototype branch
rschlaefli 4f1c0d0
docs: append prototype outcome (§12) to Mastra evaluation report
rschlaefli 76ac1af
docs(prototype): add post-prototype next-steps engineering plan
rschlaefli 579269d
feat(prototype): A1 real credit accounting (cost.ts + part.totalUsage)
rschlaefli a632df2
feat(prototype): A1 observability — native Mastra tracing via container
rschlaefli a49308f
feat(prototype): A3 background cost attribution (embeddings + summari…
rschlaefli ff04426
feat(proto/mastra-chat): A2 — route reasoning models to OpenRouter pr…
rschlaefli c2dbc0f
fix(proto/mastra-chat): skip observability container when tracing is off
rschlaefli 72036c0
feat(prototype/mastra-chat): standard OpenAI/Azure Responses API for …
rschlaefli ff673d8
docs(chat-mastra): add A2 findings + Hono integration plan, reconcile…
rschlaefli 85792ee
feat(chat-api): scaffold parallel Hono chat service + flag/Traefik wi…
rschlaefli 6e2ac0f
feat(chat-engine): extract DB-free Mastra chat engine from prototype
rschlaefli 14e5f21
feat(chat-api): Hono service hosting the Mastra chat engine (core cha…
rschlaefli 4cb046a
feat(chat): flag-gated streaming proxy to the chat-api Mastra service
rschlaefli 20d3033
fix(chat-api): make Prisma import type-only to clean the rollup bundle
rschlaefli dd4eb40
fix(chat-api): use --env-file-if-exists so the service boots under in…
rschlaefli b37e261
docs(plan): record Phase 2-4 progress + Phase 4 manual-setup next steps
rschlaefli b45cd6e
fix(chat-api): add a request body-size limit to the chat route (DoS g…
rschlaefli 51c539b
docs(plan): record final security review (1 fixed, 2 pre-existing fla…
rschlaefli 6146ebf
docs(chat): record verified Phase 4 local test-drive results
rschlaefli d754a90
test(chat-api): add openrouter smoke workflow
rschlaefli 1392f31
Merge remote-tracking branch 'origin/v3' into v3-ai
rschlaefli 1c97210
merge(v3): update v3-ai
rschlaefli 09dab97
merge(v3-ai): update mastra chat PR
rschlaefli fabe64f
fix(security): avoid token-looking test placeholder
rschlaefli 7717572
docs(project): add production-readiness review for the Mastra chat-ap…
rschlaefli File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,127 @@ | ||
| name: Chat API OpenRouter smoke | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| pull_request: | ||
| types: [opened, synchronize, reopened, ready_for_review] | ||
| paths: | ||
| - 'apps/chat-api/**' | ||
| - 'packages/chat-engine/**' | ||
| - 'packages/prisma/**' | ||
| - 'packages/prisma-data/**' | ||
| - 'pnpm-lock.yaml' | ||
| - '.github/workflows/chat-api-openrouter-smoke.yml' | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref_name }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| smoke: | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: read | ||
|
|
||
| services: | ||
| postgres: | ||
| image: postgres:15 | ||
| env: | ||
| POSTGRES_USER: klicker | ||
| POSTGRES_DB: klicker-prod | ||
| POSTGRES_HOST_AUTH_METHOD: trust | ||
| options: >- | ||
| --health-cmd pg_isready | ||
| --health-interval 10s | ||
| --health-timeout 5s | ||
| --health-retries 5 | ||
| ports: | ||
| - 5432:5432 | ||
|
|
||
| env: | ||
| APP_SECRET: local-dev-only | ||
| CHAT_MODEL_REGISTRY_JSON: >- | ||
| [{"id":"deepseek-v4-flash","deploymentId":"deepseek/deepseek-v4-flash","name":"DeepSeek V4 Flash","description":"OpenRouter smoke test model","fallback":true,"supportsReasoning":false,"supportsImageAttachments":false,"maxOutputTokens":256,"cost":{"input":0.27,"output":0.41}}] | ||
| CHAT_OPENAI_STORE_RESPONSES: 'false' | ||
| CHAT_SMOKE_API_BASE_URL: http://127.0.0.1:3305 | ||
| CHAT_SMOKE_TIMEOUT_MS: '120000' | ||
| DATABASE_URL: postgresql://klicker@localhost:5432/klicker-prod?sslmode=disable | ||
| FALLBACK_MODEL_ID: deepseek/deepseek-v4-flash | ||
| OPENAI_API_KEY: ${{ secrets.OPENROUTER_API_KEY || 'ci-placeholder' }} | ||
| OPENAI_BASE_URL: https://openrouter.ai/api/v1 | ||
| PORT: '3305' | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - uses: pnpm/action-setup@v4 | ||
| with: | ||
| version: 10.15.0 | ||
|
|
||
| - uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: 20.19.4 | ||
| cache: pnpm | ||
|
|
||
| - name: Install dependencies | ||
| run: pnpm install --frozen-lockfile | ||
|
|
||
| - name: Build smoke dependencies | ||
| run: | | ||
| pnpm --filter @klicker-uzh/prisma build | ||
| pnpm --filter @klicker-uzh/types build | ||
| pnpm --filter @klicker-uzh/grading build | ||
| pnpm --filter @klicker-uzh/util build | ||
| pnpm --filter @klicker-uzh/chat-engine build | ||
|
|
||
| - name: Prepare seeded database | ||
| run: | | ||
| pnpm --filter @klicker-uzh/prisma prisma:push:raw | ||
| pnpm --filter @klicker-uzh/prisma-data seed:raw | ||
|
|
||
| - name: Typecheck chat-api | ||
| run: pnpm --filter @klicker-uzh/chat-api check | ||
|
|
||
| - name: Start chat-api | ||
| run: | | ||
| pnpm --filter @klicker-uzh/chat-api exec tsx src/index.ts > /tmp/chat-api-openrouter-smoke.log 2>&1 & | ||
| echo "$!" > /tmp/chat-api-openrouter-smoke.pid | ||
|
|
||
| for attempt in {1..30}; do | ||
| if curl -fsS "http://127.0.0.1:${PORT}/health" >/dev/null; then | ||
| exit 0 | ||
| fi | ||
| sleep 2 | ||
| done | ||
|
|
||
| cat /tmp/chat-api-openrouter-smoke.log | ||
| exit 1 | ||
|
|
||
| - name: Run OpenRouter DeepSeek V4 Flash smoke | ||
| continue-on-error: true | ||
| run: pnpm --filter @klicker-uzh/chat-api smoke:openrouter | ||
|
|
||
| - name: Publish smoke status note | ||
| if: always() | ||
| run: | | ||
| { | ||
| echo '### Chat API OpenRouter smoke' | ||
| echo | ||
| echo 'This workflow executes the real chat-api OpenRouter smoke path against DeepSeek V4 Flash.' | ||
| echo | ||
| echo 'The smoke step is currently non-blocking because the workflow falls back to an invalid placeholder when `OPENROUTER_API_KEY` is not configured as a repository secret.' | ||
| } >> "$GITHUB_STEP_SUMMARY" | ||
|
|
||
| - name: Print chat-api logs | ||
| if: always() | ||
| run: | | ||
| if [[ -f /tmp/chat-api-openrouter-smoke.log ]]; then | ||
| cat /tmp/chat-api-openrouter-smoke.log | ||
| fi | ||
|
|
||
| - name: Stop chat-api | ||
| if: always() | ||
| run: | | ||
| if [[ -f /tmp/chat-api-openrouter-smoke.pid ]]; then | ||
| kill "$(cat /tmp/chat-api-openrouter-smoke.pid)" >/dev/null 2>&1 || true | ||
| fi | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| name: Build Docker image for mcp-student (prd) | ||
|
|
||
| on: | ||
| push: | ||
| tags: ['v*.*.*'] | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref_name }} | ||
| cancel-in-progress: true | ||
|
|
||
| env: | ||
| REGISTRY: ghcr.io | ||
| IMAGE_NAME: ${{ github.repository }}/mcp-student | ||
|
|
||
| jobs: | ||
| build-arm: | ||
| runs-on: ubuntu-24.04-arm | ||
| permissions: | ||
| contents: read | ||
| packages: write | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up QEMU | ||
| uses: docker/setup-qemu-action@v3 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Log into registry ${{ env.REGISTRY }} | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| registry: ${{ env.REGISTRY }} | ||
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Extract Docker metadata | ||
| id: meta | ||
| uses: docker/metadata-action@v5 | ||
| with: | ||
| images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-arm | ||
|
|
||
| - name: Build and push Docker image | ||
| uses: docker/build-push-action@v5 | ||
| with: | ||
| context: . | ||
| file: apps/mcp-student/Dockerfile | ||
| push: ${{ github.event_name != 'pull_request' }} | ||
| tags: ${{ steps.meta.outputs.tags }} | ||
| platforms: linux/arm64 | ||
| no-cache: true | ||
| labels: ${{ steps.meta.outputs.labels }} | ||
|
|
||
| build-amd: | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: read | ||
| packages: write | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up QEMU | ||
| uses: docker/setup-qemu-action@v3 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Log into registry ${{ env.REGISTRY }} | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| registry: ${{ env.REGISTRY }} | ||
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Extract Docker metadata | ||
| id: meta | ||
| uses: docker/metadata-action@v5 | ||
| with: | ||
| images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-amd | ||
|
|
||
| - name: Build and push Docker image | ||
| uses: docker/build-push-action@v5 | ||
| with: | ||
| context: . | ||
| file: apps/mcp-student/Dockerfile | ||
| push: ${{ github.event_name != 'pull_request' }} | ||
| tags: ${{ steps.meta.outputs.tags }} | ||
| platforms: linux/amd64 | ||
| labels: ${{ steps.meta.outputs.labels }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,106 @@ | ||
| name: Build Docker image for mcp-student (stg) | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - 'v3' | ||
| - 'v3*' | ||
| pull_request: | ||
| types: [opened, synchronize, reopened, ready_for_review] | ||
| paths: | ||
| - 'apps/mcp-student/**' | ||
| - 'packages/graphql/**' | ||
| - 'packages/types/**' | ||
| - 'packages/util/**' | ||
| - 'package.json' | ||
| - 'pnpm-lock.yaml' | ||
| - 'turbo.json' | ||
| - '.github/workflows/v3_mcp-student**' | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref_name }} | ||
| cancel-in-progress: true | ||
|
|
||
| env: | ||
| REGISTRY: ghcr.io | ||
| IMAGE_NAME: ${{ github.repository }}/mcp-student | ||
|
|
||
| jobs: | ||
| build-arm: | ||
| if: github.event_name != 'pull_request' || github.event.pull_request.draft == false | ||
| runs-on: ubuntu-24.04-arm | ||
| permissions: | ||
| contents: read | ||
| packages: write | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up QEMU | ||
| uses: docker/setup-qemu-action@v3 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Log into registry ${{ env.REGISTRY }} | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| registry: ${{ env.REGISTRY }} | ||
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Extract Docker metadata | ||
| id: meta | ||
| uses: docker/metadata-action@v5 | ||
| with: | ||
| images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-arm | ||
|
|
||
| - name: Build and push Docker image | ||
| uses: docker/build-push-action@v5 | ||
| with: | ||
| context: . | ||
| file: apps/mcp-student/Dockerfile | ||
| push: ${{ github.event_name != 'pull_request' }} | ||
| tags: ${{ steps.meta.outputs.tags }} | ||
| platforms: linux/arm64 | ||
| no-cache: true | ||
| labels: ${{ steps.meta.outputs.labels }} | ||
|
|
||
| build-amd: | ||
| if: github.event_name != 'pull_request' || github.event.pull_request.draft == false | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: read | ||
| packages: write | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up QEMU | ||
| uses: docker/setup-qemu-action@v3 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Log into registry ${{ env.REGISTRY }} | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| registry: ${{ env.REGISTRY }} | ||
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Extract Docker metadata | ||
| id: meta | ||
| uses: docker/metadata-action@v5 | ||
| with: | ||
| images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-amd | ||
|
|
||
| - name: Build and push Docker image | ||
| uses: docker/build-push-action@v5 | ||
| with: | ||
| context: . | ||
| file: apps/mcp-student/Dockerfile | ||
| push: ${{ github.event_name != 'pull_request' }} | ||
| tags: ${{ steps.meta.outputs.tags }} | ||
| platforms: linux/amd64 | ||
| labels: ${{ steps.meta.outputs.labels }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
continue-on-error: truesilently masks smoke regressionsThe smoke step is non-blocking regardless of exit code, so any regression in streaming, persistence, or credit decrement will cause the workflow to succeed and produce a green check on the PR. PRs that break the live SSE path will merge undetected until the
OPENROUTER_API_KEYsecret is wired andcontinue-on-erroris removed. Consider usingif: ${{ secrets.OPENROUTER_API_KEY != '' }}to skip the step entirely when the secret is absent, and removingcontinue-on-errorso a present secret always produces a hard gate.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!