[Bug]: Millisecond connector run IDs and unlocked state writes lose parallel MCP results #542

Description

Connector run IDs are ISO timestamps with millisecond precision. Parallel calls started in the same millisecond receive the same run ID and therefore target the same raw-data directory and fixed filenames.

Connector state is also updated through an unlocked read-modify-write. MCP discovery/tool calls read state before performing network work, then later overwrite the whole state file. Parallel calls can consequently overwrite raw results and discard each other's run history.

These are two parts of the same concurrency failure: connector operations do not have a unique run identity or serialized state commit.

Affected code

• src/connectors/io.ts:72-74: createRunId() contains no random or monotonic component.
• src/connectors/io.ts:33-57: state is read and written as a whole document without locking or compare-and-merge.
• src/connectors/io.ts:59-67: raw paths use runId plus a caller-supplied fixed filename.
• src/connectors/mcp-runtime.ts:47-67: MCP discovery reads state early, writes mcp-tools.json, then writes derived state.
• src/connectors/mcp-runtime.ts:77-120: MCP tool calls use the same pattern and fixed mcp-tool-result.json.

Steps to Reproduce

1. Freeze Date so two operations see the same millisecond.
2. Start two callMcpConnectorTool() calls for the same connector with different results using Promise.all().
3. Allow both calls to complete after both have read the original state.
4. Observe:
   • both calls report the same run ID;
   • both write <runId>/mcp-tool-result.json, so one result replaces the other;
   • each derives state from the same old snapshot, and the last state write drops the other run.

Even without a frozen clock, rapid synchronous calls frequently generate repeated timestamp IDs.

Expected Behavior

Every invocation should have a collision-resistant run ID, preserve its own immutable raw output, and merge its history/cursor update without losing concurrently completed runs.

Actual Behavior

Parallel invocations can share paths and use last-writer-wins state replacement.

Why this matters

LangChain/LangGraph agents may issue independent tool calls concurrently. Lost raw evidence breaks provenance and can make the agent read a result from the wrong invocation. Lost state history also undermines cursoring, diagnostics, and latest-run selection.

Suggested direction

• Generate IDs with crypto.randomUUID() or append a cryptographically random/monotonic suffix to the sortable timestamp.
• Serialize state updates per connector, or implement an atomic update API that re-reads and merges immediately before replacement.
• Write state through a unique temp file and atomic rename.
• Ensure run history is deduplicated by unique run ID.

Missing regression coverage

Add a fixed-clock collision test plus a parallel MCP-call test asserting distinct raw paths, preserved payloads, and both runs in state.json.

Environment

• OS: macOS
• Node.js: v24.14.1
• Affected revision: 63c848c on main
• OpenWiki version: current source checkout