test(messages): add live Claude Agent SDK smoke test against /v1/messages (#6010)

## What

Adds a live smoke test that drives the upstream
[`claude-agent-sdk`](https://github.com/anthropics/claude-agent-sdk-python)
Python package against a local OGX server, exercising the Anthropic
Messages (`/v1/messages`) path end to end through a real client. This is
the SDK companion to the Claude Code CLI smoke test merged in #5986
(RHAIENG-5377).

The SDK does not speak HTTP itself — it spawns the Claude Code CLI as a
subprocess and parses its streamed session output. So this covers a
distinct client surface (the SDK's session machinery: message streaming,
`ResultMessage` parsing) on top of the same path: full system prompt,
tool definitions, and an inline system-role message that the server must
accept and dispatch to the backing provider.

## Why live, not record/replay

The SDK drives the real CLI, which bakes the working directory, date,
and platform into every request body. The recording system keys on
SHA256 of the request body, so those hashes are not reproducible across
runs or machines — record/replay is not viable. The test runs live
against Ollama and self-skips unless **both** the `claude-agent-sdk`
package and the `claude` binary are available.

## What it asserts

Integration health, not answer quality — a small local model driving the
Claude Code harness cannot be relied on to produce specific text. The
test checks the parsed `ResultMessage`:
- `subtype == "success"` and `is_error is False`
- the backing model present in `model_usage`

A regression like a rejected inline system-role message would surface as
a session error and be caught here.

## Changes

- New test: `tests/integration/messages/test_claude_agent_sdk.py`
- New CI workflow:
`.github/workflows/integration-tests-messages-sdk.yml` (provisions
Ollama, installs the pinned CLI, runs the test live)
- `claude-agent-sdk>=0.2.0` added to the `test` dependency group (locked
to 0.2.87)

## Test plan

Ran live against local Ollama using the same CI command:

```
./scripts/integration-tests.sh \
  --stack-config server:ci-tests --setup ollama --suite messages \
  --inference-mode live --pattern test_claude_agent_sdk_smoke
```

Result:

```
tests/integration/messages/test_claude_agent_sdk.py::test_claude_agent_sdk_smoke[txt=ollama/llama3.2:3b-instruct-fp16] PASSED
1 passed, 20 deselected in 36.45s
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Charlie Doern <cdoern@redhat.com>

Author: cdoern

.github/workflows/README.md

@@ -17,7 +17,7 @@ OGX uses GitHub Actions for Continuous Integration (CI). Below is a table detail
 | Integration Auth Tests | [integration-auth-tests.yml](integration-auth-tests.yml) | Run the integration test suite with Kubernetes authentication |
 | Integration Responses, Conversations & Prompts Auth Tests | [integration-responses-conversations-auth-tests.yml](integration-responses-conversations-auth-tests.yml) | Run responses, conversations, and prompts auth tests with Kubernetes authentication |
 | SqlStore Integration Tests | [integration-sql-store-tests.yml](integration-sql-store-tests.yml) | Run the integration test suite with SqlStore |
-| Messages API - Claude Code CLI Smoke Test | [integration-tests-messages-cli.yml](integration-tests-messages-cli.yml) | Drive the real Claude Code CLI against /v1/messages (live, Ollama) |
+| Messages API - Claude Code Client Smoke Tests | [integration-tests-messages-clients.yml](integration-tests-messages-clients.yml) | Drive the Claude Code CLI and Agent SDK against /v1/messages (live, Ollama) |
 | Integration Tests (Replay) | [integration-tests.yml](integration-tests.yml) | Run the integration test suites from tests/integration in replay mode |
 | Vector IO Integration Tests | [integration-vector-io-tests.yml](integration-vector-io-tests.yml) | Run the integration test suite with various VectorIO providers |
 | OpenAPI Generator SDK Validation | [openapi-generator-validation.yml](openapi-generator-validation.yml) | Validate OpenAPI Generator SDK generation |

…flows/integration-tests-messages-cli.yml …s/integration-tests-messages-clients.yml.github/workflows/integration-tests-messages-cli.yml renamed to .github/workflows/integration-tests-messages-clients.yml .github/workflows/integration-tests-messages-cli.yml renamed to .github/workflows/integration-tests-messages-clients.yml

@@ -1,6 +1,6 @@
-name: Messages API - Claude Code CLI Smoke Test
+name: Messages API - Claude Code Client Smoke Tests
 
-run-name: Drive the real Claude Code CLI against /v1/messages (live, Ollama)
+run-name: Drive the Claude Code CLI and Agent SDK against /v1/messages (live, Ollama)
 
 on:
   push:
@@ -18,7 +18,7 @@ on:
       - 'tests/integration/messages/**'
       - 'uv.lock'
       - 'pyproject.toml'
-      - '.github/workflows/integration-tests-messages-cli.yml'
+      - '.github/workflows/integration-tests-messages-clients.yml'
       - '.github/actions/setup-test-environment/action.yml'
       - '.github/actions/run-and-record-tests/action.yml'
       - 'scripts/integration-tests.sh'
@@ -36,27 +36,31 @@ permissions:
   contents: read
 
 env:
-  # Pinned for reproducibility. The CLI bakes the cwd, date, and platform into
-  # every request body, so its traffic cannot be recorded/replayed; this test
-  # runs live against Ollama instead. Bumping only changes the client behavior
-  # under test, not any committed recordings.
+  # Pinned for reproducibility. Both clients ultimately drive the Claude Code
+  # CLI, which bakes the cwd, date, and platform into every request body, so
+  # their traffic cannot be recorded/replayed; these tests run live against
+  # Ollama instead. Bumping only changes the client behavior under test.
   CLAUDE_CODE_CLI_VERSION: '2.1.159'
+  # The Agent SDK is installed at workflow time rather than as a project
+  # dependency, since only this one live smoke test uses it; it spawns the same
+  # CLI. The SDK test self-skips if the package is unavailable.
+  CLAUDE_AGENT_SDK_VERSION: '0.2.87'
 
 jobs:
-  claude-code-cli-smoke:
-    name: Claude Code CLI smoke (ollama, live)
+  claude-code-client-smoke:
+    name: Claude Code CLI + Agent SDK smoke (ollama, live)
     runs-on: ubuntu-latest
-    # CPU-only runners generate slowly; the live CLI session can take several
-    # minutes against the Ollama model. Keep ample headroom over the test's own
-    # 600s subprocess timeout plus environment setup.
+    # CPU-only runners generate slowly; the live sessions can take several
+    # minutes against the Ollama model. Keep ample headroom over the tests' own
+    # 600s timeouts plus environment setup.
     timeout-minutes: 30
 
     steps:
       - name: Checkout repository
         uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
 
-      # Live mode (not replay) so the Ollama backend is provisioned and the
-      # real CLI traffic reaches a real model.
+      # Live mode (not replay) so the Ollama backend is provisioned and the real
+      # client traffic reaches a real model.
       - name: Setup test environment
         uses: ./.github/actions/setup-test-environment
         with:
@@ -66,17 +70,26 @@ jobs:
           suite: 'messages'
           inference-mode: 'live'
 
+      # Both clients need the Claude Code CLI at runtime: the CLI test invokes it
+      # directly, and the SDK spawns it as a subprocess.
       - name: Install Claude Code CLI
         run: |
           curl -fsSL https://claude.ai/install.sh | bash -s -- "${CLAUDE_CODE_CLI_VERSION}"
           echo "$HOME/.local/bin" >> "$GITHUB_PATH"
           "$HOME/.local/bin/claude" --version
 
-      - name: Run Claude Code CLI smoke test
+      - name: Install Claude Agent SDK
+        run: uv pip install "claude-agent-sdk==${CLAUDE_AGENT_SDK_VERSION}"
+
+      - name: Run Claude Code client smoke tests
         uses: ./.github/actions/run-and-record-tests
         with:
           stack-config: 'server:ci-tests'
           setup: 'ollama'
           suite: 'messages'
           inference-mode: 'live'
-          pattern: 'test_claude_code_cli_smoke'
+          # Selects both test_claude_code_cli_smoke and test_claude_agent_sdk_smoke.
+          # Must stay a single shell word: run-and-record-tests passes --pattern
+          # unquoted, so a -k expression with spaces would be split into separate
+          # arguments.
+          pattern: 'test_claude'

tests/integration/messages/test_claude_agent_sdk.py

@@ -0,0 +1,108 @@
+# Copyright (c) The OGX Contributors.
+# All rights reserved.
+#
+# This source code is licensed under the terms described in the LICENSE file in
+# the root directory of this source tree.
+
+"""Smoke test: drive the Claude Agent SDK against the OGX Messages API.
+
+Points the SDK at a local OGX server and runs a single prompt through the
+upstream `claude-agent-sdk` Python package (github.com/anthropics/
+claude-agent-sdk-python). The SDK does not speak HTTP itself: it spawns the
+Claude Code CLI as a subprocess and parses its streamed session output. This
+exercises a different client surface than the CLI smoke test -- the SDK's
+session machinery (message streaming, ResultMessage parsing) on top of the same
+end-to-end path through /v1/messages: full system prompt, tool definitions, and
+an inline system-role message that the server must accept and dispatch to the
+backing provider.
+
+This runs LIVE against a real backend, not in replay mode. The SDK drives the
+real CLI, which bakes the working directory, date, and platform into every
+request body, so the request-body hashes the recording system keys on are not
+reproducible across runs or machines; recording/replay is therefore not viable.
+
+The test self-skips unless both the `claude-agent-sdk` package and the `claude`
+binary are available, since the SDK requires the CLI at runtime.
+"""
+
+import asyncio
+import importlib.util
+import shutil
+
+import pytest
+
+CLAUDE_CLI = shutil.which("claude")
+HAS_SDK = importlib.util.find_spec("claude_agent_sdk") is not None
+
+pytestmark = pytest.mark.skipif(
+    CLAUDE_CLI is None or not HAS_SDK,
+    reason="claude-agent-sdk and the claude CLI must both be installed to run",
+)
+
+
+def _run_query(prompt: str, base_url: str, model: str, cwd: str) -> list:
+    """Run a single Agent SDK query() to completion and return all messages."""
+    from claude_agent_sdk import ClaudeAgentOptions, query
+
+    options = ClaudeAgentOptions(
+        model=model,
+        # Run from an isolated directory so the spawned CLI does not pick up
+        # repo-local context, which matters while permissions are bypassed.
+        cwd=cwd,
+        # Passed to the spawned CLI subprocess so it reaches OGX instead of
+        # api.anthropic.com.
+        env={
+            "ANTHROPIC_BASE_URL": base_url,
+            "ANTHROPIC_API_KEY": "dummy",
+            "ANTHROPIC_MODEL": model,
+        },
+        # The prompt is pure Q&A and triggers no tools, but bypass permissions
+        # so the non-interactive session can never stall on a permission prompt.
+        permission_mode="bypassPermissions",
+    )
+
+    messages: list = []
+
+    async def _collect() -> None:
+        async for message in query(prompt=prompt, options=options):
+            messages.append(message)
+
+    # Generous: the SDK drives the real CLI, which makes several large-context
+    # calls, and a small model on a CPU-only CI runner generates slowly (tens of
+    # seconds each). Bound it so a hung session fails loudly instead of riding
+    # the job timeout.
+    asyncio.run(asyncio.wait_for(_collect(), timeout=600))
+    return messages
+
+
+def test_claude_agent_sdk_smoke(messages_base_url, text_model_id, tmp_path):
+    """Claude Agent SDK completes a session against /v1/messages without error.
+
+    The smoke signal is integration health, not answer quality: the SDK drives a
+    full agentic session (system prompt, tools, inline system message) against
+    OGX, OGX routes it to the backing model, and the session terminates with a
+    successful ResultMessage. We deliberately do not assert on the model's text
+    output -- a small local model driving the Claude Code harness cannot be
+    relied on to produce a specific answer, but a regression like a rejected
+    system-role message (which would surface as a session error) is caught here.
+    """
+    from claude_agent_sdk import ResultMessage
+
+    prompt = "What is the capital of France? Reply with only the city name and nothing else."
+    base_url = str(messages_base_url).rstrip("/")
+
+    messages = _run_query(prompt, base_url, text_model_id, cwd=str(tmp_path))
+
+    results = [m for m in messages if isinstance(m, ResultMessage)]
+    assert results, f"Agent SDK session produced no ResultMessage; got: {[type(m).__name__ for m in messages]}"
+
+    result = results[-1]
+    assert result.subtype == "success" and not result.is_error, (
+        f"Agent SDK session reported an error talking to /v1/messages: "
+        f"subtype={result.subtype} is_error={result.is_error} errors={result.errors}"
+    )
+    # Confirm the request actually reached the backing model through /v1/messages.
+    model_usage = result.model_usage or {}
+    assert text_model_id in model_usage, (
+        f"Expected model {text_model_id} in ResultMessage.model_usage; got: {list(model_usage)}"
+    )