Skip to content

SKILL.md

Latest commit

 

History

History
112 lines (81 loc) · 5.67 KB

SKILL.md

File metadata and controls

112 lines (81 loc) · 5.67 KB
Raw
name multi-llm-orchestrator
description Orchestrates workflows across multiple LLMs with dynamic model selection at every step. Triggers on "orchestrate", "multi-model", "cross-LLM", "model comparison", or when leveraging different models' strengths.

Multi-LLM Orchestrator

Cross-model orchestration with dynamic routing. Claude Code is the conductor — manages flow, calls external LLMs via scripts/orchestrate.py, and presents human gates between steps.

Default Flow

GPT-5.4 (Analysis) → Claude Opus (Counter-analysis + Implementation) → You (Gate) → Claude Code (Execute)

This is the starting pattern, not a fixed pipeline. Any step can be rerouted at any gate.

Models

Alias Model Best for
gpt5 GPT-5.4 Analysis, structured critique, scoring
opus Claude Opus 4.6 Counter-analysis, deep reasoning, implementation
sonnet Claude Sonnet 4 Faster/cheaper Claude tasks
gemini Gemini 2.0 Pro Validation, cross-referencing, second opinions
gpt4o GPT-4o Legacy fallback

See references/model-profiles.md for strengths, weaknesses, and routing heuristics.

Prerequisites

python scripts/orchestrate.py check

Validates API keys and model reachability. If a model is unreachable, present the user with available alternatives before proceeding. For key setup see references/config-setup.md.

Running a Workflow

  1. Select workflow — Match to a template in references/workflow-templates.md, or build ad-hoc.
  2. Initialize task file — Create _orchestrate_{workflow}_{YYYY-MM-DD}_{HHmm}.md (source of truth for this run).
  3. Execute each step: a. Build a focused prompt with the Input section + relevant prior step summaries (not full prior outputs). b. Call the model: 
    python scripts/orchestrate.py call --model {model} --prompt-file /tmp/prompt.md --output step_result.md
    c. Present the result at the human gate (see Gate Protocol below). d. Route based on user's gate decision.
  4. Final step — Claude Code applies approved changes locally (no API call).

Gate Protocol

Present at every gate:

  1. Header: Which model produced this output and what it was asked.
  2. Output: <100 lines → show inline. ≥100 lines → summary inline + "Full output in {file}".
  3. Options: Approve | Bounce to {model} | Redirect to {model} | Adjust | Abort

Gate actions:

Action Effect
Approve Proceed to next step with default model
Bounce ("bounce back to gpt") Return output to the originating model with user feedback for revision
Redirect ("send to gemini") Send output to a different model, framed for that model's strength
Adjust Modify the prompt; ask user which model to use, then re-run
Abort Stop workflow, mark task file status as aborted

Ambiguous commands ("use cheaper model", "retry that"): ask the user to specify the model alias and whether they want a bounce, redirect, or adjust.

Context for Bounces and Redirects

Always include in the prompt:

  • The original input/task context (from the task file Input section)
  • The output being reviewed (full text for bounces; summary or full for redirects based on length)
  • User feedback on what to focus on, fix, or validate
  • Framing instruction: "You are reviewing and improving the previous model's work" (bounce) or "Validate/counter-analyze this output" (redirect)

Error Handling

When orchestrate.py returns an error (JSON with status, message, details):

Error Action
Auth failure (401/403) Report to user. Suggest checking API key via references/config-setup.md.
Rate limit (429) Inform user of wait time. Offer: wait, substitute model, or abort.
Server error (5xx) / timeout Inform user. Offer: retry, substitute model, skip step, or abort.

These options integrate into the gate flow — after an error, present the same gate with the error context added.

Edge Cases

  • No-op outcome: If analysis concludes no changes are needed, present that finding at the gate. User can approve (end workflow), redirect for a second opinion, or adjust the analysis scope.
  • Partial approval: If the user wants to approve some files but reject others from a rewrite, treat rejected files as an adjust — re-run with instructions specifying which files to revise.
  • Token/context overflow: If a prompt exceeds the target model's context window, split the input or summarize prior steps more aggressively. Inform the user of the truncation.
  • Malformed output: If a rewrite is missing file markers, has duplicate filenames, or omits files, flag the issue at the gate and offer: adjust (re-run with explicit format instructions), bounce, or redirect.
  • Execution failure: If package-all.sh fails after writing files, report the error. The .bak files enable manual rollback. Do not auto-rollback.
  • Missing prior task data: If a prior task file lacks a score or is incomplete, note this when presenting prior run history and default to "start fresh."

Task File

Each run gets one task file as its log. Format and schema defined in references/workflow-templates.md.

Reference Files

File Purpose When consulted
references/workflow-templates.md Workflow definitions, task file schema Every run
references/model-profiles.md Model strengths/weaknesses for routing decisions When choosing models
references/config-setup.md API key setup, adding models/providers Setup and troubleshooting
references/api-patterns.md Script internals, provider request formats Debugging failures only