Skip to content

fix(package/ai): fix Gateway image cost reporting bug #10478

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
R-Taneja wants to merge 4 commits into from
Closed

fix(package/ai): fix Gateway image cost reporting bug #10478

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e31ac18
Fix cost reporting
R-Taneja Nov 22, 2025
4015546
Changeset
R-Taneja Nov 22, 2025
21bac77
Second pass
R-Taneja Nov 22, 2025
67c1b45
Vade fix
R-Taneja Nov 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/rich-chicken-walk.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'ai': patch
---

fix(package/ai): fix Gateway image cost reporting bug
24 changes: 20 additions & 4 deletions packages/ai/src/generate-image/generate-image.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -200,13 +200,29 @@ Only applicable for HTTP-based providers.
if (providerName === 'gateway') {
const currentEntry = providerMetadata[providerName];
if (currentEntry != null && typeof currentEntry === 'object') {
providerMetadata[providerName] = {
const merged = {
Copy link
Copy Markdown
Collaborator

@shaper shaper Nov 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is gateway-specific logic, ideally there would be a way to solve this on the Gateway server or provider side. Earlier in the flow, summing things before we get into this code.

Since this is just reporting, and I get that I think the issue is this ai-core code is the only place we have all of the data come together, can we report the gateway provider metadata for each chunk/image-response within that chunk/response and leave it up to the caller to sum or aggregate or look across it if they choose to do so?

Copy link
Copy Markdown
Collaborator

@gr2m gr2m Nov 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The whole if (providerName === 'gateway') {} block is already gateway specific and needs to be refactored.

Copy link
Copy Markdown
Contributor Author

@R-Taneja R-Taneja Nov 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that the whole block needs to be refactored. As a temporary solution to ship image models asap, 2 paths I see are:

  1. Go with my code in an earlier commit that sums the cost and sets the cost field to the true total cost
  2. Go with the current code in the PR that adds an allCalls array to the provider metadata containing the metadata for each request, and let the user sum the costs if they want. The only real issue with this is the metadata output gets quite long, and can be somewhat confusing because the data outside the allCalls array is just for the last request and shows the cost of one image (same issue), so if a user doesn’t read into the allCalls array they won’t realize what the actual cost is.

I was leaning towards option 1, but I realized that other fields other than cost are also being overwritten with the last request's metadata. This could be an issue as we'd only return fields like modelAttempts, generationId, resolvedProvider, etc for the last request if n > 1. So maybe option 2 is the best because it returns metadata from all calls. I'm just concerned about the duplicative-ness of the last call's metadata being outside allCalls (in the top level providerMetadata object) and also in the last position of the allCalls array.

...(currentEntry as object),
...metadata,
} as ImageModelV3ProviderMetadata[string];
} else {
} as Record<string, unknown>;
const existingCalls =
(currentEntry as Record<string, unknown>)['allCalls'];
const callsArray = Array.isArray(existingCalls)
? (existingCalls as unknown[])
: [];
(merged as Record<string, unknown>)['allCalls'] = [
...callsArray,
metadata,
];
providerMetadata[providerName] =
metadata as ImageModelV3ProviderMetadata[string];
merged as ImageModelV3ProviderMetadata[string];
} else {
const first = { ...(metadata as object) } as Record<
string,
unknown
>;
(first as Record<string, unknown>)['allCalls'] = [metadata];
providerMetadata[providerName] =
first as ImageModelV3ProviderMetadata[string];
}
const imagesValue = (
providerMetadata[providerName] as { images?: unknown }
Expand Down
Loading