Skip to content

feat(openai): attach Dify app_id as request metadata (opt-in)#3203

Draft
mas-sakai wants to merge 9 commits into
langgenius:mainfrom
mas-sakai:feat/openai-app-id-metadata
Draft

feat(openai): attach Dify app_id as request metadata (opt-in)#3203
mas-sakai wants to merge 9 commits into
langgenius:mainfrom
mas-sakai:feat/openai-app-id-metadata

Conversation

@mas-sakai

@mas-sakai mas-sakai commented May 26, 2026

Copy link
Copy Markdown

Summary

Adds opt-in support for attaching the Dify app_id as OpenAI request metadata, so usage on the OpenAI Usage Dashboard can be filtered per Dify app.

When the new enable_request_metadata credential is set to enabled, the plugin reads app_id from the current Dify session (via get_current_session() from the SDK) and attaches {dify_app_id, dify_source} as the metadata field on both chat.completions.create and responses.create. The default is disabled, so behavior is unchanged unless the operator opts in.

This is the OpenAI counterpart of the same feature already shipped for Vertex AI (#3168) and Bedrock (#3201). The responsibility split mirrors those plugins:

Plugin What the plugin attaches What the operator must enable
Vertex AI labels on generateContent BigQuery billing export
Bedrock requestMetadata on Converse CloudWatch invocation logging
OpenAI metadata on Chat / Responses Stored Completions on the OpenAI account — the plugin enables store=true automatically

Design note: store is set to true alongside metadata

E2E verification against a live OpenAI account showed the API rejects metadata when store is false:

BadRequestError: The 'metadata' parameter is only allowed when 'store' is enabled.

Enabling the metadata feature therefore inherently requires store=true. The plugin makes this explicit: apply_dify_metadata_if_enabled sets store=true alongside the metadata (a non-destructive merge that respects an explicit store value already on the request). This means requests and responses are persisted to Stored Completions on the OpenAI account when the operator opts in. The credential's help text in provider/openai.yaml documents this storage behavior in both English and Simplified Chinese, and links to the OpenAI store reference.

Bedrock (#3201) and Vertex AI (#3168) do not have this constraint, so their plugins remain unchanged.

Related:

Note

Draft / dependency. pyproject.toml temporarily pins dify_plugin to the fork branch ryuta-kobayashi-ug/dify-plugin-sdks@feat/pass-session-to-model-plugins because it carries the get_current_session() API used here (langgenius/dify-plugin-sdks#313). The pin will revert to a versioned dify_plugin spec once that SDK change merges and a release ships.

Change Type

  • Documentation / non-plugin change
  • Non-LLM plugin (tools, extensions, datasource, etc.)
  • LLM plugin

Screenshots / Videos

Before After
Usage Dashboard rows are anonymous; per-app filtering is impossible. With opt-in enabled, the plugin sets store=true and attaches dify_app_id and dify_source, so requests are recorded to Stored Completions and can be filtered per app.

End-to-end screenshots will be added once the dependent SDK change (#313) ships and a Dify deployment can be exercised against a live OpenAI account with Stored Completions enabled.

LLM Plugin Checklist

Areas affected by this change
  • Message flow (system messages, user ↔ assistant turn-taking)
  • Tool interaction flow (multi-round usage, Agent App and Agent Node)
  • Multimodal input (images, PDFs, audio, video, etc.)
  • Multimodal output (images, audio, video, etc.)
  • Structured output (JSON, XML, etc.)
  • Token consumption metrics
  • Other LLM functionality (reasoning, grounding, prompt caching, etc.) — request metadata for billing/observability
  • New models / model parameter fixes

The change is additive and behind a credential that defaults to disabled. No existing message flow, tool, multimodal, structured-output, or token-accounting code path is altered. Only _chat_generate and _build_responses_api_params gain a one-line opt-in hook that mutates the request kwargs immediately before the OpenAI client call.

Version

  • Bumped top-level version in manifest.yaml (0.4.10.4.3)
  • dify_plugin declared in pyproject.toml and locked in uv.lock (currently pinned to the SDK branch carrying fix getnumtoken retrun list[int] #313 — see Draft note above)

Testing

  • Unit tests — uv run pytest tests/ (22 passed): covers normalization (UUID passthrough, punctuation, mixed case, non-ASCII, 512-char truncation, empty string, non-string coercion), build_dify_metadata (None / empty / source marker / UUID passthrough / length normalization), and apply_dify_metadata_if_enabled (no-op on missing or disabled credential, silent on session-lookup failure, non-destructive metadata merge, and the new store=true behavior: sets store=true on opt-in, preserves an explicit caller-supplied store, and never touches store when disabled).
  • Local deployment — Dify version: pending (blocked on SDK fix getnumtoken retrun list[int] #313 release; will retest after the pin is reverted)
  • SaaS (cloud.dify.ai)

mas-sakai and others added 5 commits May 26, 2026 13:31
Add models/llm/_metadata.py with normalize_metadata_value,
build_dify_metadata, and apply_dify_metadata_if_enabled helpers,
plus unit tests. Constraints match OpenAI's metadata spec
(<=16 pairs, key <=64, value <=512, strings). No character pattern
restriction is documented, so normalize only stringifies and
truncates.

The helper deliberately does not touch the store parameter; whether
metadata surfaces in the Usage Dashboard is governed by the account's
Stored Completions setting, which is the terminus owner's responsibility.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…s routes

Insert apply_dify_metadata_if_enabled at two call sites:
- _chat_generate, before client.chat.completions.create, mutating
  extra_model_kwargs.
- _build_responses_api_params, so both _chat_generate_responses_api
  and _chat_generate_responses_api_stream receive metadata via the
  shared param builder. _build_responses_api_params now accepts an
  optional credentials kwarg threaded from both callers.

Both routes are covered because the api_protocol credential lets users
pick chat or responses per provider/model. Legacy _generate
(completions.create) is intentionally out of scope.

Session lookup is wrapped in a broad try/except — metadata is
best-effort telemetry and must never break generation if the SDK is
missing or the session context is not initialized.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add enable_request_metadata to both model_credential_schema and
provider_credential_schema as a select (enabled/disabled, default
disabled), mirroring the existing api_protocol structure. Labels,
options, and help text include en_US and zh_Hans.

The help text documents that this plugin does not set store=true,
so dashboard visibility requires the OpenAI account to have Stored
Completions enabled separately.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Point dify_plugin at the fork branch
ryuta-kobayashi-ug/dify-plugin-sdks@feat/pass-session-to-model-plugins
to obtain get_current_session() in model plugins. uv.lock refreshed.
Adds pytest as a dev dependency with the matching pytest config used
by other plugins (testpaths=tests, pythonpath=.).

Once the upstream SDK release ships, the pin will revert to the
versioned dify_plugin spec.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311, langgenius/dify-plugin-sdks#313

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces an optional feature to attach Dify metadata (such as dify_app_id and dify_source) to OpenAI Chat Completions and Responses API requests, enabling per-app filtering in the OpenAI Usage Dashboard. The changes include a new helper module _metadata.py for metadata normalization and injection, integration within the LLM generation workflows, updated provider credential schemas, and a suite of unit tests. The review feedback highlights opportunities to improve type safety by using Any for coerced inputs, fix a logical inconsistency where falsy non-string values (like 0) are dropped, prevent overwriting existing metadata by merging dictionaries, and add a corresponding unit test.

Comment thread models/openai/models/llm/_metadata.py Outdated
Comment thread models/openai/models/llm/_metadata.py Outdated
Comment thread models/openai/models/llm/_metadata.py Outdated
Comment thread models/openai/models/llm/_metadata.py Outdated
Comment thread models/openai/tests/test_metadata.py
…adata per review

- Widen `normalize_metadata_value` / `build_dify_metadata` type hints to
  `Any` to match the str() coercion the implementations already do.
- Reject only `None` and `""` in `build_dify_metadata`; other falsy
  values (e.g. numeric 0) flow through to normalization rather than
  being silently dropped, matching the design intent for the helper.
- Merge into existing `target["metadata"]` instead of overwriting it
  when the caller has already populated the field; fall back to direct
  assignment if the existing value is not a dict.
- Add tests covering the merge-with-existing path, the non-dict
  fallback, and the falsy-but-non-empty input case.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…malize per review

Per gemini-code-assist on langgenius#3233. Aligns 4 plugins on the same
'no side effects on existing args' principle.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
E2E verified that the OpenAI API rejects `metadata` when store is false
(BadRequestError: "The 'metadata' parameter is only allowed when 'store'
is enabled."). The metadata feature therefore inherently requires
store=true.

apply_dify_metadata_if_enabled now sets store=true alongside the metadata
(respecting an explicit store value already on the request), and the
credential help text + inline comments document that requests/responses
are persisted to Stored Completions on the OpenAI account.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mas-sakai mas-sakai deployed to models/openai June 9, 2026 07:56 — with GitHub Actions Active
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant