Skip to content

Commit 2ec2f28

Browse files
franciscojavierarceofranciscojavierarceo
authored
feat: Add Responses memory read path (#6162)
## Summary - adds `MemoryToolConfig` to Responses requests and read-side `MemoryConfig` to the built-in Responses provider - gates Memory Alpha behind `MemoryConfig.enabled=False` by default so server operators must explicitly opt in - retrieves memory from the configured vector store only with the memory metadata filter and a required owner filter - resolves owner scope from the authenticated principal when present, so request metadata cannot spoof another owner - keeps vector-store ABAC on the `openai_search_vector_store` path and treats denied memory reads as no injected memory - injects retrieved memory as hidden system context, without file-search outputs, annotations, or citations - bounds retrieved memory context by result count and approximate token budget - regenerates API schemas, Stainless schemas, and provider docs for the read-side memory surface ## Test plan - `uv run pytest tests/unit/providers/responses/builtin/test_openai_responses_memory.py -q` -> 13 passed - `uv run pytest tests/unit/api/test_responses_memory_models.py tests/unit/providers/responses/builtin/test_openai_responses_memory.py -q` -> 16 passed - `uv run pre-commit run --all-files` -> passed ## Stack - PR 1: #6162 read path - PR 2: #6163 write path and Memory Alpha docs --------- Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
1 parent 737c445 commit 2ec2f28

16 files changed

Lines changed: 1005 additions & 10 deletions

File tree

‎client-sdks/stainless/openapi.yml‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12216,6 +12216,13 @@ components:
1221612216
type: array
1221712217
- type: 'null'
1221812218
description: Context management configuration. When set with type 'compaction', automatically compacts conversation history when token count exceeds the compact_threshold.
12219+
memory:
12220+
anyOf:
12221+
- $ref: '#/components/schemas/MemoryToolConfig'
12222+
title: MemoryToolConfig
12223+
- type: 'null'
12224+
description: Controls owner-scoped memory retrieval for this response.
12225+
title: MemoryToolConfig
1221912226
additionalProperties: true
1222012227
type: object
1222112228
required:
@@ -12839,6 +12846,53 @@ components:
1283912846
- name
1284012847
title: MCPListToolsTool
1284112848
description: Tool definition returned by MCP list tools operation.
12849+
MemoryToolConfig:
12850+
properties:
12851+
enabled:
12852+
type: boolean
12853+
title: Enabled
12854+
description: Whether to retrieve memory context for this response.
12855+
default: true
12856+
owner_id:
12857+
anyOf:
12858+
- type: string
12859+
- type: 'null'
12860+
description: Stable owner identifier used to scope memory retrieval.
12861+
vector_store_id:
12862+
anyOf:
12863+
- type: string
12864+
- type: 'null'
12865+
description: Memory vector store override. Defaults to the server memory vector store.
12866+
max_num_results:
12867+
anyOf:
12868+
- type: integer
12869+
maximum: 50.0
12870+
minimum: 1.0
12871+
- type: 'null'
12872+
description: Maximum memory chunks to retrieve.
12873+
max_context_tokens:
12874+
anyOf:
12875+
- type: integer
12876+
minimum: 1.0
12877+
- type: 'null'
12878+
description: Approximate token budget for injected memory context.
12879+
filters:
12880+
anyOf:
12881+
- additionalProperties: true
12882+
type: object
12883+
- type: 'null'
12884+
description: "Additional vector-store filters combined with the required owner filter. Use the vector store filter shape, such as {'type': 'eq', 'key': ..., 'value': ...} or {'type': 'and', 'filters': [...]}."
12885+
ranking_options:
12886+
anyOf:
12887+
- $ref: '#/components/schemas/SearchRankingOptions'
12888+
title: SearchRankingOptions
12889+
- type: 'null'
12890+
description: Ranking options for memory vector-store search.
12891+
title: SearchRankingOptions
12892+
additionalProperties: false
12893+
type: object
12894+
title: MemoryToolConfig
12895+
description: Configuration for Responses memory retrieval.
1284212896
MessageBatch:
1284312897
properties:
1284412898
id:

‎docs/docs/providers/responses/inline_builtin.mdx‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,14 @@ Be concise, structured, and focused on helping the next LLM seamlessly continue
9191
| `compaction_config.default_compact_threshold` | `int \| None` | No | | Default token threshold for auto-compaction via context_management. If set, conversations exceeding this token count will be automatically compacted. |
9292
| `compaction_config.tokenizer_encoding` | `str \| None` | No | | Default tiktoken encoding name for token counting (e.g. 'o200k_base', 'cl100k_base'). Applied as a server-level default after any per-request override via extra_body. If not set, encoding is resolved from the model name via tiktoken, then model-family prefix mappings, then character-based estimation. |
9393
| `compaction_config.model_tokenizer_mappings` | `dict[str, str]` | No | &#123;'llama': 'cl100k_base', 'mistral': 'cl100k_base', 'claude': 'cl100k_base', 'gemma': 'cl100k_base', 'qwen': 'cl100k_base', 'phi': 'cl100k_base', 'deepseek': 'cl100k_base'&#125; | Map model name prefixes to tiktoken encoding names. Used as a heuristic fallback when tiktoken cannot resolve the model name directly. Matching is case-insensitive on the model name after stripping any provider prefix (e.g., 'ollama/llama3.2:3b' matches the 'llama' prefix). Admins can extend this to support custom or fine-tuned models. |
94+
| `memory_config` | `MemoryConfig` | No | enabled=False default_vector_store_id=None owner_metadata_key='owner_id' memory_metadata_key='memory' max_num_results=5 max_context_tokens=1200 read_prompt_template='These are concise summaries of previous conversations for this owner. Use them only as contextual recall. They may be stale or incomplete. Do not mention them as search results or cite them.' | Configuration for Responses memory reads. |
95+
| `memory_config.enabled` | `bool` | No | False | Enable Responses memory reads. Disabled by default because memory is an alpha feature. |
96+
| `memory_config.default_vector_store_id` | `str \| None` | No | | Default vector store containing conversation memory files. |
97+
| `memory_config.owner_metadata_key` | `str` | No | owner_id | Vector-store file attribute key used for owner scoping. |
98+
| `memory_config.memory_metadata_key` | `str` | No | memory | Vector-store file attribute key used to identify memory files. |
99+
| `memory_config.max_num_results` | `int` | No | 5 | Default maximum memory chunks to retrieve. |
100+
| `memory_config.max_context_tokens` | `int` | No | 1200 | Default approximate token budget for injected memory context. |
101+
| `memory_config.read_prompt_template` | `str` | No | These are concise summaries of previous conversations for this owner. Use them only as contextual recall. They may be stale or incomplete. Do not mention them as search results or cite them. | Prompt text that frames retrieved memory context. |
94102
| `moderation_endpoint` | `str \| None` | No | | URL of an OpenAI-compatible /v1/moderations endpoint for guardrails. The endpoint must accept POST &#123;"input": "text"&#125; and return &#123;"results": [&#123;"flagged": bool, "categories": &#123;...&#125;&#125;]&#125;. |
95103
| `moderation_headers` | `dict[str, str] \| None` | No | | HTTP headers to send with moderation endpoint requests. Use this to provide authentication for hosted moderation services (e.g., &#123;'Authorization': 'Bearer sk-...'&#125;). These headers are server-side only and never exposed to clients. |
96104

‎docs/static/ogx-spec.yaml‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11285,6 +11285,13 @@ components:
1128511285
type: array
1128611286
- type: 'null'
1128711287
description: Context management configuration. When set with type 'compaction', automatically compacts conversation history when token count exceeds the compact_threshold.
11288+
memory:
11289+
anyOf:
11290+
- $ref: '#/components/schemas/MemoryToolConfig'
11291+
title: MemoryToolConfig
11292+
- type: 'null'
11293+
description: Controls owner-scoped memory retrieval for this response.
11294+
title: MemoryToolConfig
1128811295
additionalProperties: true
1128911296
type: object
1129011297
required:
@@ -11526,6 +11533,53 @@ components:
1152611533
- name
1152711534
title: MCPListToolsTool
1152811535
description: Tool definition returned by MCP list tools operation.
11536+
MemoryToolConfig:
11537+
properties:
11538+
enabled:
11539+
type: boolean
11540+
title: Enabled
11541+
description: Whether to retrieve memory context for this response.
11542+
default: true
11543+
owner_id:
11544+
anyOf:
11545+
- type: string
11546+
- type: 'null'
11547+
description: Stable owner identifier used to scope memory retrieval.
11548+
vector_store_id:
11549+
anyOf:
11550+
- type: string
11551+
- type: 'null'
11552+
description: Memory vector store override. Defaults to the server memory vector store.
11553+
max_num_results:
11554+
anyOf:
11555+
- type: integer
11556+
maximum: 50.0
11557+
minimum: 1.0
11558+
- type: 'null'
11559+
description: Maximum memory chunks to retrieve.
11560+
max_context_tokens:
11561+
anyOf:
11562+
- type: integer
11563+
minimum: 1.0
11564+
- type: 'null'
11565+
description: Approximate token budget for injected memory context.
11566+
filters:
11567+
anyOf:
11568+
- additionalProperties: true
11569+
type: object
11570+
- type: 'null'
11571+
description: "Additional vector-store filters combined with the required owner filter. Use the vector store filter shape, such as {'type': 'eq', 'key': ..., 'value': ...} or {'type': 'and', 'filters': [...]}."
11572+
ranking_options:
11573+
anyOf:
11574+
- $ref: '#/components/schemas/SearchRankingOptions'
11575+
title: SearchRankingOptions
11576+
- type: 'null'
11577+
description: Ranking options for memory vector-store search.
11578+
title: SearchRankingOptions
11579+
additionalProperties: false
11580+
type: object
11581+
title: MemoryToolConfig
11582+
description: Configuration for Responses memory retrieval.
1152911583
MessageBatch:
1153011584
properties:
1153111585
id:

‎docs/static/stainless-ogx-spec.yaml‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12216,6 +12216,13 @@ components:
1221612216
type: array
1221712217
- type: 'null'
1221812218
description: Context management configuration. When set with type 'compaction', automatically compacts conversation history when token count exceeds the compact_threshold.
12219+
memory:
12220+
anyOf:
12221+
- $ref: '#/components/schemas/MemoryToolConfig'
12222+
title: MemoryToolConfig
12223+
- type: 'null'
12224+
description: Controls owner-scoped memory retrieval for this response.
12225+
title: MemoryToolConfig
1221912226
additionalProperties: true
1222012227
type: object
1222112228
required:
@@ -12839,6 +12846,53 @@ components:
1283912846
- name
1284012847
title: MCPListToolsTool
1284112848
description: Tool definition returned by MCP list tools operation.
12849+
MemoryToolConfig:
12850+
properties:
12851+
enabled:
12852+
type: boolean
12853+
title: Enabled
12854+
description: Whether to retrieve memory context for this response.
12855+
default: true
12856+
owner_id:
12857+
anyOf:
12858+
- type: string
12859+
- type: 'null'
12860+
description: Stable owner identifier used to scope memory retrieval.
12861+
vector_store_id:
12862+
anyOf:
12863+
- type: string
12864+
- type: 'null'
12865+
description: Memory vector store override. Defaults to the server memory vector store.
12866+
max_num_results:
12867+
anyOf:
12868+
- type: integer
12869+
maximum: 50.0
12870+
minimum: 1.0
12871+
- type: 'null'
12872+
description: Maximum memory chunks to retrieve.
12873+
max_context_tokens:
12874+
anyOf:
12875+
- type: integer
12876+
minimum: 1.0
12877+
- type: 'null'
12878+
description: Approximate token budget for injected memory context.
12879+
filters:
12880+
anyOf:
12881+
- additionalProperties: true
12882+
type: object
12883+
- type: 'null'
12884+
description: "Additional vector-store filters combined with the required owner filter. Use the vector store filter shape, such as {'type': 'eq', 'key': ..., 'value': ...} or {'type': 'and', 'filters': [...]}."
12885+
ranking_options:
12886+
anyOf:
12887+
- $ref: '#/components/schemas/SearchRankingOptions'
12888+
title: SearchRankingOptions
12889+
- type: 'null'
12890+
description: Ranking options for memory vector-store search.
12891+
title: SearchRankingOptions
12892+
additionalProperties: false
12893+
type: object
12894+
title: MemoryToolConfig
12895+
description: Configuration for Responses memory retrieval.
1284212896
MessageBatch:
1284312897
properties:
1284412898
id:

‎src/ogx/core/server/server.py‎

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222
from fastapi.exceptions import RequestValidationError
2323
from fastapi.responses import JSONResponse
2424
from openai import BadRequestError
25+
from packaging.version import InvalidVersion, Version
2526
from starlette.types import ASGIApp, Receive, Scope, Send
2627

2728
from ogx.core.access_control.access_control import AccessDeniedError
@@ -243,21 +244,27 @@ async def __call__(self, scope: Scope, receive: Receive, send: Send) -> Any:
243244
client_version = headers.get(b"x-ogx-client-version", b"").decode()
244245
if client_version:
245246
try:
246-
client_version_parts = tuple(map(int, client_version.split(".")[:2]))
247-
server_version_parts = tuple(map(int, self.server_version.split(".")[:2]))
248-
if client_version_parts != server_version_parts:
247+
if not _client_version_is_compatible(client_version, self.server_version):
249248
return await _send_error_response(
250249
send,
251250
status=httpx.codes.UPGRADE_REQUIRED,
252251
message=f"Client version {client_version} is not compatible with server version {self.server_version}. Please update your client.",
253252
)
254-
except (ValueError, IndexError):
253+
except InvalidVersion:
255254
# If version parsing fails, let the request through
256255
pass
257256

258257
return await self.app(scope, receive, send)
259258

260259

260+
def _client_version_is_compatible(client_version: str, server_version: str) -> bool:
261+
client = Version(client_version)
262+
server = Version(server_version)
263+
if client.is_devrelease or server.is_devrelease or client.local or server.local:
264+
return True
265+
return (client.major, client.minor) == (server.major, server.minor)
266+
267+
261268
class ProviderDataMiddleware:
262269
"""Middleware to set up request context for all routes.
263270

‎src/ogx/providers/inline/responses/builtin/config.py‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,12 @@
3131
"use the information in this summary to assist with your own analysis:"
3232
)
3333

34+
DEFAULT_MEMORY_READ_PROMPT = (
35+
"These are concise summaries of previous conversations for this owner. Use them "
36+
"only as contextual recall. They may be stale or incomplete. Do not mention them "
37+
"as search results or cite them."
38+
)
39+
3440

3541
class CompactionConfig(BaseModel):
3642
"""Configuration for conversation compaction behavior and prompt templates."""
@@ -106,6 +112,42 @@ def validate_model_tokenizer_mappings(cls, v: dict[str, str]) -> dict[str, str]:
106112
return v
107113

108114

115+
class MemoryConfig(BaseModel):
116+
"""Configuration for Responses memory reads."""
117+
118+
enabled: bool = Field(
119+
default=False,
120+
description="Enable Responses memory reads. Disabled by default because memory is an alpha feature.",
121+
)
122+
default_vector_store_id: str | None = Field(
123+
default=None,
124+
description="Default vector store containing conversation memory files.",
125+
)
126+
owner_metadata_key: str = Field(
127+
default="owner_id",
128+
description="Vector-store file attribute key used for owner scoping.",
129+
)
130+
memory_metadata_key: str = Field(
131+
default="memory",
132+
description="Vector-store file attribute key used to identify memory files.",
133+
)
134+
max_num_results: int = Field(
135+
default=5,
136+
ge=1,
137+
le=50,
138+
description="Default maximum memory chunks to retrieve.",
139+
)
140+
max_context_tokens: int = Field(
141+
default=1200,
142+
ge=1,
143+
description="Default approximate token budget for injected memory context.",
144+
)
145+
read_prompt_template: str = Field(
146+
default=DEFAULT_MEMORY_READ_PROMPT,
147+
description="Prompt text that frames retrieved memory context.",
148+
)
149+
150+
109151
class ResponsesPersistenceConfig(BaseModel):
110152
"""Nested persistence configuration for the responses provider."""
111153

@@ -127,6 +169,11 @@ class BuiltinResponsesImplConfig(BaseModel):
127169
description="Configuration for conversation compaction behavior and prompt templates",
128170
)
129171

172+
memory_config: MemoryConfig = Field(
173+
default_factory=MemoryConfig,
174+
description="Configuration for Responses memory reads.",
175+
)
176+
130177
moderation_endpoint: str | None = Field(
131178
default=None,
132179
description="URL of an OpenAI-compatible /v1/moderations endpoint for guardrails. "

‎src/ogx/providers/inline/responses/builtin/impl.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,7 @@ async def initialize(self) -> None:
107107
vector_stores_config=self.config.vector_stores_config,
108108
connectors_api=self.connectors_api,
109109
compaction_config=self.config.compaction_config,
110+
memory_config=self.config.memory_config,
110111
)
111112
await self.openai_responses_impl.initialize()
112113

@@ -166,6 +167,7 @@ async def create_openai_response(
166167
extra_body=request.model_extra,
167168
stream_options=request.stream_options,
168169
context_management=request.context_management,
170+
memory=request.memory,
169171
)
170172
return result
171173

0 commit comments

Comments
 (0)