feat(agent): add local skills and MCP tools

- move discovered skills to Lummy-owned paths and add /new slash UI
- surface tool timeout/retry metadata and assistant fallback on tool failure
- add stdio MCP discovery/runtime docs and tests

Author: outlndrr

README.md

@@ -192,6 +192,7 @@ Notes:
 - UI talks to same-origin backend routes, so no CORS setup needed
 - backend still requires `LUMMY_OPENROUTER_API_KEY`
 - UI opens live session SSE stream at `/api/v1/sessions/:id/events`
+- typing `/` opens slash commands for `/new` and discovered `/skill:<name>` entries
 - UI includes debug pane for HTTP actions, SSE status, and streamed events
 - backend now logs request + run/provider/tool lifecycle to Erlang logger
 
@@ -234,7 +235,9 @@ Notes:
 | `LUMMY_AGENT_ALLOWED_TOOL_CAPABILITIES` | `math,filesystem_read,network,external_api,memory` | Default capability allowlist; blocks `file_write` and `shell_command` by default |
 | `LUMMY_AGENT_MAX_TOOL_RISK` | `low` | Global tool risk ceiling |
 | `LUMMY_AGENT_EXTERNAL_PROVIDER_MAX_TOOL_RISK` | `low` | Extra risk ceiling applied when non-`basic-local` provider is in chain |
-| `LUMMY_SKILLS_PATH` | `./skills` | Directory of local JSON skill manifests |
+| `LUMMY_SKILLS_PATH` | `./skills` | Directory of local Agent Skills packages (`<name>/SKILL.md`) used by explicit `skill_ids` |
+| `LUMMY_AGENT_SKILL_DISCOVERY_PATH` | `./skills` | Directory scanned for discovered slash/model-visible skills; relative paths resolve under `LUMMY_AGENT_TOOL_FILE_ROOT` |
+| `LUMMY_MCP_SERVERS_PATH` | `./mcp_servers.json` | Local MCP server config file |
 
 ## Dev
 
@@ -260,6 +263,8 @@ Or from `cmd.exe` / PowerShell in project root use bundled `rebar3.cmd` wrapper.
 - Domain core notes: `docs/domain-core.md`
 - Persistence notes: `docs/persistence.md`
 - Session runtime notes: `docs/session-runtime.md`
+- Skills v1: `docs/skills.md`
+- MCP v1: `docs/mcp.md`
 - Quickstart: `docs/quickstart.md`
 - HTTP API notes: `docs/http-api.md`
 - Event stream notes: `docs/event-stream.md`

TODO.md

@@ -28,22 +28,35 @@
 - [x] add `skill_ids` on run start
 - [x] persist effective skill config on `AgentRun`
 - [x] merge prompt/provider/tool policy deterministically
-- [ ] add registry abstraction and richer manifest consumption (`memory_hints`, `tags`)
-- [ ] document stable merge rules + sample manifests
+- [x] add registry abstraction and richer manifest consumption (`memory_hints`, `tags`)
+- [x] document stable merge rules + sample manifests
+- [x] align local skill packaging with Agent Skills `SKILL.md` spec
 
 ### M4. MCP v1
-- [ ] add stdio MCP client lifecycle
-- [ ] discover MCP tools into local tool definitions
-- [ ] keep local capability/risk mapping authoritative
-- [ ] execute MCP tools through current tool-call lifecycle
-- [ ] add timeout/cancel/retry tests
+- [ ] add session-scoped stdio MCP client lifecycle
+- [x] discover MCP tools into local tool definitions
+- [x] keep local capability/risk mapping authoritative
+- [x] execute MCP tools through current tool-call/runtime path
+- [ ] translate MCP input schemas/result payloads more faithfully
+- [ ] add explicit cancel/retry coverage beyond timeout path
 
 ### M5. Safety groundwork after M1-M4
 - [ ] API auth groundwork
 - [ ] secret storage groundwork
 - [ ] audit log groundwork
 - [ ] quotas / budgets groundwork
 
+## Improvements pass (from IMPROVEMENTS.md)
+- [x] add slash command `/new` in UI
+- [x] constrain slash menu height, add scroll, bump menu text size
+- [x] support multiline `description: >` skill frontmatter values
+- [x] stop discovered-skill lookup from reading Pi paths; use Lummy config/env path instead
+- [x] fix explicit skill use path so discovered skills load reliably
+- [x] add assistant fallback message when tool call fails
+- [x] verify tool retries/timeouts and cover with tests
+- [x] reduce duplicate snapshot refresh at end of agent run
+- [x] clean up touched code/docs/tests
+
 ## Notes
 - Removed planning docs after folding execution order into this file.
-- Current implementation pass completed M1 and M2, plus core M3 skill plumbing.
+- Current implementation pass completed M1, M2, M3, and partial M4.

docs/http-api.md

@@ -15,6 +15,8 @@ Copy-paste examples live in `docs/quickstart.md`.
 - `POST /api/v1/providers/openai-compatible/verify`
 
 ### Session and run API
+- `GET /api/v1/skills`
+- `GET /api/v1/sessions`
 - `POST /api/v1/sessions`
 - `GET /api/v1/sessions/:id`
 - `POST /api/v1/sessions/:id/messages`
@@ -42,7 +44,13 @@ Copy-paste examples live in `docs/quickstart.md`.
 - message append routes through `session_manager` -> `session_actor`
 - run start routes through `session_manager` -> `session_actor`
 - higher-level basic agent runs route through `basic_agent` orchestration
-- start-run request supports per-run execution policy and skill selection (`skill_ids` map to Agent Skills `name` / directory names under `LUMMY_SKILLS_PATH`):
+- `GET /api/v1/skills` returns discovered skills from `LUMMY_AGENT_SKILL_DISCOVERY_PATH` resolved relative to `tools.file_root` when needed
+- message append accepts `metadata` string fields; explicit callers can invoke discovered skills by setting:
+  - `lummy_agent.skill: <name>` or `skill: <name>`
+  - `lummy_agent.skills: comma,separated,names` or `skills: comma,separated,names`
+- `/ui` exposes discovered skills through a generic slash menu: typing `/` routes to `/new` and `/skill:<name>` entries with descriptions
+- explicit user skill invocations are expanded into the user prompt even when hidden from model-visible auto invocation
+- start-run request supports per-run execution policy and Agent Skills selection (`skill_ids` map to Agent Skills `name` / directory names under `LUMMY_SKILLS_PATH` / `skills.path` compatibility root):
   - `provider_fallback_ids`
   - `skill_ids`
   - `allowed_tools`
@@ -56,13 +64,15 @@ Copy-paste examples live in `docs/quickstart.md`.
 - `apply-result` persists tool output, appends tool message, and resumes run back to `calling_model`
 - `respond` appends assistant message and completes run from the model-response boundary
 - run JSON includes `provider_fallback_ids`, `tool_policy`, `skill_ids`, `skill_prompt_append`, `skill_memory_hints`, `skill_tags`, `plan_tasks`, and `scratchpad`
-- tool call list route reads persisted tool calls for a run
+- chat messages preserve metadata, allowing UI to surface user skill commands and agent-used skill reads in the transcript
+- tool call list route reads persisted tool calls for a run and includes runtime `timeout_ms`, `max_retries`, and `max_attempts`
 - SSE routes honor `Last-Event-ID` for persisted replay before live streaming
 - without `Last-Event-ID`, SSE routes stay live-only
 - event-log routes return persisted envelopes as JSON for debugging
 - event-log routes accept optional `after=<sequence>` query filtering
 - session memory route returns stored memory entries plus scored recall matches
 - session memory route accepts optional `query=<text>` and `limit=<n>`
+- basic agent tool failures fall back to an assistant message in the snapshot instead of returning a bare route error when the run can be completed with an explanation
 - responses are JSON only for now
 
 ## Runtime wiring
@@ -74,8 +84,7 @@ Copy-paste examples live in `docs/quickstart.md`.
 ## Current limitations
 - no websocket API yet
 - no auth on API routes yet
-- no session list endpoint yet
-- no pagination controls on event-log routes yet
+- no pagination controls on session list or event-log routes yet
 - no attachment / artifact upload surface yet
 - simple Lustre UI exists at `GET /ui`, consumes session SSE directly, and shows a debug pane for live events
 

docs/mcp.md

@@ -0,0 +1,57 @@
+# MCP v1
+
+MCP support currently enters lummy as tool source.
+
+## Config
+
+- `LUMMY_MCP_SERVERS_PATH` defaults to `./mcp_servers.json`
+- if config file is missing, no MCP servers are loaded
+
+## Server config format
+
+```json
+{
+  "servers": [
+    {
+      "server_id": "fake",
+      "command": "uvx mcp-server-example",
+      "working_directory": ".",
+      "tool_mappings": [
+        {
+          "tool_name": "search",
+          "capabilities": ["external_api"],
+          "risk": "low",
+          "timeout_ms": 5000
+        }
+      ]
+    }
+  ]
+}
+```
+
+## Local policy rules
+
+Local config is source of truth.
+
+- only tools present in `tool_mappings` are exposed
+- unmapped discovered tools are ignored
+- local `capabilities` decide capability policy checks
+- local `risk` decides risk policy checks
+- local `timeout_ms` drives invocation timeout
+- tool names are namespaced as `mcp:<server_id>:<tool_name>`
+
+## Runtime behavior
+
+- tool discovery calls MCP `tools/list` over stdio
+- tool invocation calls MCP `tools/call` over stdio
+- discovered MCP tools are merged with built-in tools for run execution
+- run policy is extended with mapped MCP tool names, then normal capability/risk checks still apply
+- timeout handling reuses existing tool worker timeout path
+
+## Current limits
+
+- stdio only
+- one-off stdio handshake per discovery/call today; no long-lived client actor yet
+- generic object schema exposed to provider today; MCP input schema is not yet translated through
+- content extraction currently keeps text content only from `tools/call` result
+- no prompts/resources/sampling support yet