The bundled filesystem MCP server is pinned to @modelcontextprotocol/server-filesystem@2025.11.25 (in agent-environment's mcp_server_template.json, image ghcr.io/scaleapi/mcp-atlas:1.2.5). This version returns structured content for directory_tree that fails MCP output validation, so every directory_tree call errors out regardless of the model.
Error
MCP error -32602: Output validation error: Invalid structured content for tool directory_tree:
[ { "expected": "string", "code": "invalid_type", "path": ["content"], ... } ]
We observed this on a large fraction of tasks that route through filesystem_directory_tree during evaluation (dozens of occurrences across runs), affecting all evaluated models equally.
Root cause (fixed upstream)
This is upstream issue modelcontextprotocol/servers#3110 — "directory_tree returns structured content array instead of string" — closed on 2025-12-11 (fix PR #3113). The pinned 2025.11.25 predates the fix; it is included in 2025.12.18 and later (latest 2026.1.14).
Request
Bump the filesystem pin in the harness/image to @modelcontextprotocol/server-filesystem@2025.12.18 (or latest 2026.1.14). Since this is a pure server-side output-validation bug (not a model capability signal), bumping restores those tasks for all submissions while keeping the benchmark standardized.
The bundled filesystem MCP server is pinned to @modelcontextprotocol/server-filesystem@2025.11.25 (in agent-environment's mcp_server_template.json, image ghcr.io/scaleapi/mcp-atlas:1.2.5). This version returns structured content for directory_tree that fails MCP output validation, so every directory_tree call errors out regardless of the model.
Error
MCP error -32602: Output validation error: Invalid structured content for tool directory_tree:
[ { "expected": "string", "code": "invalid_type", "path": ["content"], ... } ]
We observed this on a large fraction of tasks that route through filesystem_directory_tree during evaluation (dozens of occurrences across runs), affecting all evaluated models equally.
Root cause (fixed upstream)
This is upstream issue modelcontextprotocol/servers#3110 — "directory_tree returns structured content array instead of string" — closed on 2025-12-11 (fix PR #3113). The pinned 2025.11.25 predates the fix; it is included in 2025.12.18 and later (latest 2026.1.14).
Request
Bump the filesystem pin in the harness/image to @modelcontextprotocol/server-filesystem@2025.12.18 (or latest 2026.1.14). Since this is a pure server-side output-validation bug (not a model capability signal), bumping restores those tasks for all submissions while keeping the benchmark standardized.