refactor!: remove Safety API and replace with moderation_endpoint (#5291)

## Summary

Remove the entire Safety API subsystem and replace it with a single
`moderation_endpoint` config field on the responses provider. Guardrails
are now a simple boolean toggle.

### Before
```
Client → POST /v1/moderations → SafetyRouter → ShieldsRoutingTable → Provider → external service
Client → Responses API (guardrails=["llama-guard"]) → safety_api.run_moderation() → same chain
```

### After
```
Client → Responses API (guardrails=true) → httpx.post(moderation_endpoint) → external service
```

No proxy endpoint, no routing table, no provider abstraction, no model
IDs. One config field, one boolean, one HTTP call.

### What's removed
- **`Api.safety` and `Api.shields`** — enum values, protocol
definitions, routing tables, routers
- **`/v1/moderations`** — standalone endpoint (users call external
moderation services directly)
- **`/v1/shields`** — shield introspection endpoints
- **`/v1/safety/run-shield`** — native shield endpoint
- **All safety providers** — `inline::llama-guard`,
`inline::code-scanner`, `inline::prompt-guard`, `remote::bedrock`,
`remote::nvidia`, `remote::sambanova`, `remote::passthrough`
- **Safety infrastructure** — `SafetyRouter`, `ShieldsRoutingTable`,
`SafetyConfig`, `ShieldStore`, `ShieldToModerationMixin`,
`ShieldRunnerMixin`, `ResponseGuardrailSpec`
- **Shield registration endpoints** — `POST /v1/shields`, `DELETE
/v1/shields/{identifier}`

### What's added
- **`moderation_endpoint`** on `BuiltinResponsesImplConfig` — URL of an
OpenAI-compatible `/v1/moderations` endpoint. Documented contract:
accepts `POST {"input": "text"}`, returns `{"results": [{"flagged":
bool, "categories": {...}}]}`
- **Response format validation** — logs a warning with a link to the
OpenAI moderations docs if the endpoint returns an unexpected format
- **`guardrails: bool`** — simplified from a list of model IDs to a
boolean toggle via `extra_body`
- **`x-extra-body-field`** — new OpenAPI generator mechanism to keep
`guardrails` typed server-side while hidden from the public schema

### Configuration

```yaml
providers:
  responses:
  - provider_id: builtin
    provider_type: inline::builtin
    config:
      moderation_endpoint: "https://api.openai.com/v1/moderations"
```

```python
client.responses.create(
    model="gpt-4o",
    input="Hello",
    extra_body={"guardrails": True},
)
```

### Rationale

- `/v1/moderations` as a passthrough adds a network hop for zero value —
clients can call any moderation service directly
- The only server-side value is guardrails during generation
(mid-generation moderation checks aren't composable from the client).
This is now a direct HTTP call
- 6 safety providers with different auth, config, and edge cases is too
much maintenance. Users point `moderation_endpoint` at OpenAI (free),
Azure, or any compatible service
- Letting clients pass URLs in requests would be an SSRF vector, so
`moderation_endpoint` is server-config only

## Test plan

- [x] `uv run pre-commit run --all-files` passes
- [x] Distribution codegen passes
- [x] Provider codegen passes
- [x] OpenAPI spec generation passes (46 paths, 65 operations)
- [x] Unit tests fixed and passing (streaming guardrail test, lazy
imports, env var tests)
- [x] OpenAI coverage baseline updated (92.9%)
- [ ] Full CI (integration tests)

BREAKING CHANGE: Safety API removed entirely. `/v1/moderations`,
`/v1/shields`, `/v1/safety/run-shield` endpoints removed. All safety
providers removed. `guardrails` field changed from list to boolean.
Configure `moderation_endpoint` on the responses provider config.

---------

Signed-off-by: Sébastien Han <seb@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: leseb

.github/workflows/integration-tests.yml

@@ -109,7 +109,7 @@ jobs:
         id: changed-files
         if: github.event_name == 'pull_request'
         run: |
-          CHANGED=$(gh pr diff ${{ github.event.pull_request.number }} --name-only)
+          CHANGED=$(gh api --paginate "repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/files" --jq '.[].filename')
           # Pass as a single-line escaped string for the argument
           {
             echo "files<<EOF"

AGENTS.md

@@ -7,7 +7,7 @@ Human contributors should follow the conventions in CONTRIBUTING.md.
 ## Project Overview
 
 OGX is an API server implementing the OpenAI Responses API, Chat Completions,
-Embeddings, and supporting APIs (files, vector stores, batches, eval, safety). It supports
+Embeddings, and supporting APIs (files, vector stores, batches, eval, and responses guardrails). It supports
 multiple inference backends (OpenAI, Azure, Bedrock, vLLM, Ollama, WatsonX, etc.) through
 a provider architecture.
 

ARCHITECTURE.md

@@ -4,7 +4,7 @@ This document describes the internal architecture of OGX for contributors and AI
 
 ## System Overview
 
-OGX is a server that exposes a unified API for AI capabilities: inference, agents, safety, vector storage, evaluation, and more. It is provider-agnostic: the same API works whether the backend is Ollama, OpenAI, vLLM, Fireworks, or dozens of other services.
+OGX is a server that exposes a unified API for AI capabilities: inference, responses orchestration, vector storage, tool execution, evaluation, and more. It is provider-agnostic: the same API works whether the backend is Ollama, OpenAI, vLLM, Fireworks, or dozens of other services.
 
 The codebase is split into two packages:
 
@@ -30,7 +30,7 @@ Route Dispatch
   v
 Router  (src/ogx/core/routers/)
   |
-  |-- Looks up the resource (model, shield, etc.) in the RoutingTable
+  |-- Looks up the resource (model, vector store, tool group, etc.) in the RoutingTable
   |-- Resolves which provider handles this resource
   |-- Enforces access control policies
   |
@@ -80,7 +80,7 @@ Each provider spec declares:
 
 ### Provider Registry
 
-`src/ogx/providers/registry/` contains one file per API (e.g., `inference.py`, `safety.py`). Each file defines an `available_providers()` function that returns all `ProviderSpec` objects for that API. The registry is loaded at startup by `get_provider_registry()` in `core/distribution.py`.
+`src/ogx/providers/registry/` contains one file per API (e.g., `inference.py`, `responses.py`). Each file defines an `available_providers()` function that returns all `ProviderSpec` objects for that API. The registry is loaded at startup by `get_provider_registry()` in `core/distribution.py`.
 
 ### Provider Resolution
 
@@ -105,22 +105,18 @@ Api.models (RoutingTable)  <-->  Api.inference (Router)
 
 The full list of auto-routed pairs is defined in `builtin_automatically_routed_apis()` in `core/distribution.py`:
 
-| Routing Table API     | Router API       |
-|-----------------------|------------------|
-| `Api.models`          | `Api.inference`  |
-| `Api.shields`         | `Api.safety`     |
-| `Api.datasets`        | `Api.datasetio`  |
-| `Api.scoring_functions` | `Api.scoring`  |
-| `Api.benchmarks`      | `Api.eval`       |
-| `Api.tool_groups`     | `Api.tool_runtime` |
-| `Api.vector_stores`   | `Api.vector_io`  |
+| Routing Table API   | Router API         |
+|---------------------|--------------------|
+| `Api.models`        | `Api.inference`    |
+| `Api.tool_groups`   | `Api.tool_runtime` |
+| `Api.vector_stores` | `Api.vector_io`    |
 
 ## The API Layer (`ogx_api`)
 
 The `ogx_api` package defines all public-facing types and protocols:
 
-- **Protocols** -- Python `Protocol` classes like `Inference`, `Safety` that define the API contract. HTTP routes are defined via FastAPI routers in `fastapi_routes.py` modules.
-- **Data Types** -- Pydantic models for requests, responses, and resources (e.g., `Model`, `Shield`, `ChatCompletionRequest`).
+- **Protocols** -- Python `Protocol` classes like `Inference`, `Responses` that define the API contract. HTTP routes are defined via FastAPI routers in `fastapi_routes.py` modules.
+- **Data Types** -- Pydantic models for requests, responses, and resources (e.g., `Model`, `VectorStore`, `ChatCompletionRequest`).
 - **Provider Specs** -- `InlineProviderSpec`, `RemoteProviderSpec`, and related types that define how providers are declared.
 - **Internal utilities** -- KVStore and SqlStore abstract interfaces live here so third-party providers can use them without depending on the full server.
 
@@ -171,7 +167,7 @@ Used by: inference store (chat completion logs), conversations, prompts.
 
 ### Distribution Registry
 
-`src/ogx/core/store/` implements `DistributionRegistry`, which tracks all registered resources (models, shields, datasets, etc.) across providers. It persists to the configured KVStore so resources survive server restarts.
+`src/ogx/core/store/` implements `DistributionRegistry`, which tracks all registered resources (models, vector stores, tool groups, prompts, etc.) across providers. It persists to the configured KVStore so resources survive server restarts.
 
 ## Configuration
 
@@ -184,19 +180,15 @@ version: 2
 distro_name: starter
 apis:
   - inference
-  - agents
-  - safety
+  - responses
+  - vector_io
   # ...
 providers:
   inference:
     - provider_id: ollama
       provider_type: remote::ollama
       config:
         base_url: ${env.OLLAMA_URL:=http://localhost:11434/v1}
-  safety:
-    - provider_id: llama-guard
-      provider_type: inline::llama-guard
-      config: {}
 storage:
   type: sqlite
   db_path: ...
@@ -264,14 +256,13 @@ src/
   ogx_api/          # API definitions package (separate pip package)
     inference/              # Inference protocol, models, FastAPI routes
     responses/              # Responses API protocol and routes
-    safety/                 # Safety protocol and routes
     datatypes.py            # Shared data types
     providers/              # Provider spec types
     internal/               # KVStore/SqlStore interfaces
   ogx/              # Server implementation
     core/
       server/               # FastAPI server, auth, routing
-      routers/              # API-specific routers (inference, safety, etc.)
+      routers/              # API-specific routers (inference, responses, etc.)
       routing_tables/       # Resource-to-provider mapping
       storage/              # KVStore and SqlStore backends
       store/                # Distribution registry

benchmarking/k8s-benchmark/apply.sh

@@ -15,7 +15,6 @@ export POSTGRES_DB=ogx
 export POSTGRES_PASSWORD=ogx
 
 export INFERENCE_MODEL=meta-llama/Llama-3.2-3B-Instruct
-export SAFETY_MODEL=meta-llama/Llama-Guard-3-1B
 
 export BENCHMARK_INFERENCE_MODEL=$INFERENCE_MODEL
 export OGX_WORKERS=4

benchmarking/k8s-benchmark/stack-configmap.yaml

@@ -7,8 +7,6 @@ data:
     - agents
     - files
     - inference
-    - files
-    - safety
     - tool_runtime
     - vector_io
     providers:
@@ -32,43 +30,29 @@ data:
         config:
           storage_dir: ${env.FILES_STORAGE_DIR:=~/.ogx/distributions/starter/files}
           metadata_store:
-            type: sqlite
-            db_path: ${env.SQLITE_STORE_DIR:=~/.ogx/distributions/starter}/files_metadata.db
+            table_name: files_metadata
+            backend: sql_default
       vector_io:
       - provider_id: ${env.ENABLE_CHROMADB:+chromadb}
         provider_type: remote::chromadb
         config:
           url: ${env.CHROMADB_URL:=}
-          kvstore:
-            type: postgres
-            host: ${env.POSTGRES_HOST:=localhost}
-            port: ${env.POSTGRES_PORT:=5432}
-            db: ${env.POSTGRES_DB:=ogx}
-            user: ${env.POSTGRES_USER:=ogx}
-            password: ${env.POSTGRES_PASSWORD:=ogx}
-      safety:
-      - provider_id: llama-guard
-        provider_type: inline::llama-guard
-        config:
-          excluded_categories: []
+          persistence:
+            namespace: vector_io::chroma_remote
+            backend: kv_default
       agents:
       - provider_id: builtin
         provider_type: inline::builtin
         config:
-          persistence_store:
-            type: postgres
-            host: ${env.POSTGRES_HOST:=localhost}
-            port: ${env.POSTGRES_PORT:=5432}
-            db: ${env.POSTGRES_DB:=ogx}
-            user: ${env.POSTGRES_USER:=ogx}
-            password: ${env.POSTGRES_PASSWORD:=ogx}
-          responses_store:
-            type: postgres
-            host: ${env.POSTGRES_HOST:=localhost}
-            port: ${env.POSTGRES_PORT:=5432}
-            db: ${env.POSTGRES_DB:=ogx}
-            user: ${env.POSTGRES_USER:=ogx}
-            password: ${env.POSTGRES_PASSWORD:=ogx}
+          persistence:
+            agent_state:
+              namespace: agents
+              backend: kv_default
+            responses:
+              table_name: responses
+              backend: sql_default
+              max_write_queue_size: 10000
+              num_writers: 4
       tool_runtime:
       - provider_id: brave-search
         provider_type: remote::brave-search
@@ -118,25 +102,24 @@ data:
         prompts:
           backend: kv_default
           namespace: prompts
-    models:
-    - metadata:
-        embedding_dimension: 768
-      model_id: nomic-embed-text-v1.5
-      provider_id: sentence-transformers
-      model_type: embedding
-    - metadata: {}
-      model_id: Qwen/Qwen3-Reranker-0.6B
-      provider_id: transformers
-      model_type: rerank
-    - model_id: ${env.INFERENCE_MODEL}
-      provider_id: vllm-inference
-      model_type: llm
-    shields:
-    - shield_id: ${env.SAFETY_MODEL:=meta-llama/Llama-Guard-3-1B}
-    vector_dbs: []
-    datasets: []
-    scoring_fns: []
-    benchmarks: []
+    registered_resources:
+      models:
+      - metadata:
+          embedding_dimension: 768
+        model_id: nomic-embed-text-v1.5
+        provider_id: sentence-transformers
+        model_type: embedding
+      - metadata: {}
+        model_id: Qwen/Qwen3-Reranker-0.6B
+        provider_id: transformers
+        model_type: rerank
+      - model_id: ${env.INFERENCE_MODEL}
+        provider_id: vllm-inference
+        model_type: llm
+      vector_dbs: []
+      datasets: []
+      scoring_fns: []
+      benchmarks: []
     server:
       port: 8323
 kind: ConfigMap

benchmarking/k8s-benchmark/stack-k8s.yaml.template

@@ -40,16 +40,12 @@ spec:
           value: "5432"
         - name: INFERENCE_MODEL
           value: "${INFERENCE_MODEL}"
-        - name: SAFETY_MODEL
-          value: "${SAFETY_MODEL}"
         - name: TAVILY_SEARCH_API_KEY
           value: "${TAVILY_SEARCH_API_KEY}"
         - name: VLLM_URL
           value: http://vllm-server.default.svc.cluster.local:8000/v1
         - name: VLLM_MAX_TOKENS
           value: "3072"
-        - name: VLLM_SAFETY_URL
-          value: http://vllm-server-safety.default.svc.cluster.local:8001/v1
         - name: VLLM_TLS_VERIFY
           value: "false"
         - name: OGX_LOGGING

benchmarking/k8s-benchmark/stack_run_config.yaml

@@ -4,8 +4,6 @@ apis:
 - agents
 - files
 - inference
-- files
-- safety
 - tool_runtime
 - vector_io
 providers:
@@ -39,11 +37,6 @@ providers:
       persistence:
         namespace: vector_io::chroma_remote
         backend: kv_default
-  safety:
-  - provider_id: llama-guard
-    provider_type: inline::llama-guard
-    config:
-      excluded_categories: []
   agents:
   - provider_id: builtin
     provider_type: inline::builtin
@@ -120,8 +113,6 @@ registered_resources:
   - model_id: ${env.INFERENCE_MODEL}
     provider_id: vllm-inference
     model_type: llm
-  shields:
-  - shield_id: ${env.SAFETY_MODEL:=meta-llama/Llama-Guard-3-1B}
   vector_dbs: []
   datasets: []
   scoring_fns: []

benchmarking/rag/config.yaml

@@ -90,7 +90,6 @@ storage:
       backend: kv_default
 registered_resources:
   models: []
-  shields: []
   vector_dbs: []
 server:
   port: 8321
@@ -154,6 +153,4 @@ vector_stores:
     default_timeout_seconds: 120
     default_max_concurrency: 3
     max_document_tokens: 100000
-safety:
-  default_shield_id: null
 connectors: []

client-sdks/openapi/templates/python/lib/cli/ogx_client.py

@@ -33,7 +33,6 @@ def _get_version():
 from .post_training import post_training
 from .providers import providers
 from .scoring_functions import scoring_functions
-from .shields import shields
 from .toolgroups import toolgroups
 from .vector_stores import vector_stores
 
@@ -92,7 +91,6 @@ def ogx_client(ctx, endpoint: str, api_key: str, config: str | None):
 # Register all subcommands
 ogx_client.add_command(models, "models")
 ogx_client.add_command(vector_stores, "vector_stores")
-ogx_client.add_command(shields, "shields")
 ogx_client.add_command(eval_tasks, "eval_tasks")
 ogx_client.add_command(providers, "providers")
 ogx_client.add_command(datasets, "datasets")