docs(project): add codeapi integration research and feature plans

[rschlaefli]

What This Adds

This PR adds four planning documents to project/plans_future/ that design how KlickerUZH can use the DF self-hosted sandboxed code-execution service (codeapi, already live in production for DF LibreChat):

1. RESEARCH-codeapi-integration.md — shared research base: the sandbox capability envelope (5 runtimes, no egress, no runtime installs, 5-min wall), the Bearer-JWT auth contract and required claims, the exec/upload/download API shapes, a recommended Klicker identity mapping (tenant_id = one Klicker-wide tenant, sub = participantId), classroom load limits, and evidence that code execution helps tutoring accuracy (PAL ICML 2023, Khanmigo).
2. PLAN-code-element-type.md — deep plan for a new ElementType.CODE: students write Python in a CodeMirror editor, submissions run against instructor-defined test cases in the sandbox, weighted partial credit flows through the existing evaluation machinery. Grading goes through a new async seam (PENDING → Hatchet durable task → subscription/poll) because the current synchronous grading transaction cannot host a 1–300s sandbox call. Includes a full touchpoint checklist across ~10 packages, surface staging (practice quiz + microlearning first, live quiz deferred), 7 implementation slices, and a 10–15 dev-day estimate.
3. PLAN-chat-code-execution-tool.md — native execute_code tool for course chatbots (deliberately NOT an MCP server), targeting the Mastra chat stack (PR feat(chat): add Mastra chat-api prototype and OpenRouter smoke #5126 apps/chat-api + packages/chat-engine): Mastra createTool injected via the engine's AgentExtras.tools seam, per-chatbot enableCodeExecution flag gated host-side, credit cost fold-in following the imageDescriptionCost pattern (which survives in chat-api), verify-and-guide prompt contract, tool UI via the existing makeAssistantToolUI pattern. Includes an explicit "why inline await, not Hatchet" rationale. 5 slices, 3–5 dev-day estimate. The pre-Mastra (direct apps/chat route) variant is preserved in git history (832579e56).
4. PLAN-codeapi-further-features.md — ranked follow-on candidates (rehypePrism syntax highlighting first, group-activity code tasks, instructor authoring aid, tutor-runs-student-code bridge) with explicit exclusions and an overall sequencing recommendation.

Important Details

• Docs only — no code, schema, or dependency changes.
• All v3 file:line references were verified against d6c7772f8; Mastra-stack references were verified against codex/mastra-chat-openrouter-smoke (PR feat(chat): add Mastra chat-api prototype and OpenRouter smoke #5126, 2026-07-12).
• The docs were independently reviewed before the first commit (4-lens review verifying file:line claims, cross-doc consistency, and an adversarial pass on the async-grading architecture, plus a second cross-model review). All findings were integrated; the review record is in the research doc. Highest-value catches: the PENDING grading state does not fit the current QuestionResponse schema invariants (persistence model is now an explicit slice-1 gate), worker-side grading finalization must cover all side effects of today's grading transaction (shared finalizeQuestionResponse helper), and the student client needs a reload-safe pending-state machine.
• The second commit (201af9263) retargets the chat-tool plan at the Mastra migration: same architecture and estimates, new seam/file refs, shared packages/ codeapi-client module (consumed by chat-api now, the CODE element's Hatchet worker later), and a sequencing note to build after PR feat(chat): add Mastra chat-api prototype and OpenRouter smoke #5126 merges. The CODE element plan is unaffected by the migration (note added).
• Placed in plans_future/ (speculative, no implementation branch) per the existing project/ conventions.

Branch Coverage

• Base: v3
• Head: 201af9263
• Reviewed: 2 commits (initial plans 832579e56 + Mastra retarget 201af9263), 4 files, 426 insertions(+) vs v3, docs only.

Review Focus

• Whether the async grading seam (PENDING → Hatchet → push/poll) is the right architectural direction before any implementation starts.
• Whether the chat tool's placement on the Mastra chat-api stack (build after PR feat(chat): add Mastra chat-api prototype and OpenRouter smoke #5126) and the inline-await-not-Hatchet call match your expectations.
• Whether the v1 scoping calls are right: practice quiz + microlearning only, Python only, per-element test cases, live quiz deferred.
• The open questions listed at the end of each doc (persistence model, resubmission policy, pilot course, infra prerequisites).

Verification

Current head:

• git diff --stat origin/v3...HEAD -> 4 files changed, 426 insertions(+), all under project/plans_future/
• Mastra-stack claims spot-verified against codex/mastra-chat-openrouter-smoke (agent.ts tool seam, chat-api tool merge + maxSteps + credit fold-in, mcp.ts namespacing) and Mastra docs (createTool context.abortSignal, toModelOutput)

Earlier branch verification:

• Independent pre-commit review (two reviewers) on 832579e56 -> findings integrated, record in RESEARCH-codeapi-integration.md; the Mastra retarget was a targeted docs revision without a second external round (noted in the review record)

Not run:

• No build/test impact (documentation only).

Blocking Before Merge

None (docs only). Merge whenever the plans are agreed; implementation would start on separate branches per plan.

Follow-Up After Merge

• Resolve infra open questions with DF (JWKS key for a Klicker tenant, principal_source allow-list decision, sandbox package-set audit) before starting the CODE element build.
• Start the chat-tool build only after the Mastra chat-api PR feat(chat): add Mastra chat-api prototype and OpenRouter smoke #5126 merges (or consciously fall back to the pre-Mastra design in git history).
• Quick win independent of codeapi: enable rehypePrism in packages/markdown (dep already present, import commented out).

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ad9bac67-9697-4dbf-90b1-658e45dc31c3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[greptile-apps]

Confidence Score: 5/5

This looks safe to merge.

• No blocking issues found in the changed docs.

Important Files Changed

Filename	Overview
project/plans_future/PLAN-chat-code-execution-tool.md	Adds the Mastra-targeted plan for a native chatbot code-execution tool.
project/plans_future/PLAN-code-element-type.md	Adds the future CODE element plan with async grading, persistence gates, UI scope, and implementation slices.
project/plans_future/PLAN-codeapi-further-features.md	Adds ranked follow-on feature candidates and sequencing for codeapi-related work.
project/plans_future/RESEARCH-codeapi-integration.md	Adds shared research on codeapi behavior, limits, auth, APIs, and integration assumptions.

_{Reviews (2): Last reviewed commit: "docs(project): retarget chat code-execut..." | Re-trigger Greptile}

[greptile-apps]

JWT Claims Stay Incomplete

The tool template only passes sub into mintCodeapiJwt, while the research contract requires claims such as iss, aud, jti, iat, nbf, exp, principal_source, and auth_context_hash. If the implementation follows this snippet literally, every POST /v1/exec call can be rejected by codeapi authentication instead of returning sandbox output.

Context Used: CLAUDE.md (source)