Skip to content

README.md

Latest commit

 

History

History
141 lines (106 loc) · 6.47 KB

README.md

File metadata and controls

141 lines (106 loc) · 6.47 KB
Raw

dolphin-mcp-pilot

A production-ready MCP server for Apache DolphinScheduler.

dolphin-mcp-pilot exposes 53+ tools for projects, workflows, DAG creation, schedules, instances, resources, logs, monitoring and raw API passthrough — designed for AI agents that need to operate DolphinScheduler beyond basic read-only usage.

🎯 Why this project?

Most public DolphinScheduler MCP servers only cover basic read/list/start/stop scenarios. This project is designed for real operations work:

  • ✅ Create SQL / DAG workflows in one line
  • ✅ Manage schedules (create / online / offline / delete)
  • ✅ Control process instances (pause / resume / rerun / rerun-from-failure)
  • ✅ View task logs, force task success / skip failed task
  • ✅ Manage resources (view/update content)
  • ✅ Roll back workflow versions, clone workflows
  • ✅ Use raw API as a safety valve
  • ✅ Support multi-tenant per-request auth

🚀 Key features

  • 53+ tools covering most practical DS operations
  • Two auth modes: API Token (X-DS-Token) or User/Password (X-DS-User + X-DS-Password)
  • Multi-tenant HTTP mode: each caller can use its own credentials
  • MCP 2.0 stateless HTTP with automatic compatibility for MCP 1.x clients
  • Workflow creation: simple SQL and complex DAG workflows with multiple task types
  • Schedule management (cron-based)
  • Instance lifecycle control (pause/resume/rerun/rerun-from-failure/delete)
  • Resource content management and version rollback / workflow clone
  • Raw API passthrough for uncovered edge cases

🚀 Quick Start

Prerequisites

  • A running DolphinScheduler 3.x instance whose API is reachable from Docker
  • Docker with Compose v2 (docker compose version)
  • A DolphinScheduler API token (recommended), or a username and password
# 1. Clone the repository
git clone https://github.com/iflytek/dolphin-mcp-pilot.git
cd dolphin-mcp-pilot

# 2. Configure environment
cp .env.example .env
# Edit .env — set DS_URL and DS_TOKEN (or DS_USER/DS_PASSWORD)
# Example DS_URL: http://your-dolphinscheduler-host:12345/dolphinscheduler

# 3. Build and start the service from this checkout
docker compose --profile dev up -d dolphin-mcp-pilot-dev

# 4. Confirm that the container is healthy
docker compose --profile dev ps

The MCP endpoint is now http://localhost:8001/mcp/ (the trailing slash is required). Add it to an HTTP/SSE-capable MCP client:

{
  "mcpServers": {
    "dolphinscheduler": {
      "type": "sse",
      "url": "http://localhost:8001/mcp/",
      "headers": { "X-DS-Token": "your_api_token" }
    }
  }
}

As a safe first check, ask your agent: “List my DolphinScheduler projects and workflows. Do not make any changes.” For client-specific configuration and username/password auth, see Client Config.

💡 Common use cases

Scenario Example request Main tools
Investigate a failed run “Find the latest failed workflow, show the failed task and its log, and suggest the next action without changing anything.” ds_list_process_instances, ds_list_task_instances, ds_get_latest_failure_log
Backfill missing data “Backfill 2026-08-01 through 2026-08-07 serially, starting from the validation task and including downstream tasks.” ds_complement_data
Create and schedule a workflow “Create a daily SQL workflow, add its cron schedule, and show me the definition before putting it online.” ds_create_workflow, ds_set_schedule, ds_online_schedule
Give multiple agents controlled access Run one HTTP MCP service while each caller supplies its own DolphinScheduler credentials. Per-request X-DS-* headers

The tools can also pause, resume, rerun, clone, and roll back workflows; manage resources; and fall back to raw DolphinScheduler APIs for uncovered operations. Start with ds_help(category="quickstart") inside your MCP client to discover the recommended workflow for each task.

📚 Documentation

Document Description
📦 Installation Docker Compose (dev/prod), from source, as package, run modes
⚙️ Configuration Environment variables, auth options, Compose tunables
🚀 Deployment Production deployment, Compose reference, verify, troubleshoot
📊 Features Feature comparison table, tool categories
🔐 Client Config MCP client setup (CodeBuddy, Claude Desktop, etc.), multi-tenant auth
📖 API Reference All 53+ tools, parameter conventions, error handling (中文)
❓ FAQ Common issues and solutions (中文)

✨ What's new

  • MCP 2.0: supports the stateless 2026-07-28 protocol while keeping legacy handshake clients and stdio configurations working.
  • Guided troubleshooting: ds_list_process_instances attaches a next_action hint to RUNNING/FAILURE instances, pointing agents to ds_list_task_instances to inspect individual task nodes.
  • Reliable backfill ordering: serial complement uses the complementStartDate/complementEndDate range format so DolphinScheduler generates instances in strict day-by-day order.
  • Flexible task params: ds_update_task_param accepts both snake_case and camelCase field names and reports ignored fields.

🤝 Contributing

Contributions are welcome. See CONTRIBUTING.md for project changes, or follow the example contribution guide to share a tested MCP client configuration.

Used dolphin-mcp-pilot for something real? Write it up in cases/ — a gallery of community usage stories (agent-driven DolphinScheduler ops), each linked to a public post.

📄 License

Apache-2.0

🙏 Acknowledgments

Built with the official MCP Python SDK and inspired by the Apache DolphinScheduler community.