fix(package/ai): fix Gateway image cost reporting bug#10478
fix(package/ai): fix Gateway image cost reporting bug#10478
Conversation
| const currentEntry = providerMetadata[providerName]; | ||
| if (currentEntry != null && typeof currentEntry === 'object') { | ||
| providerMetadata[providerName] = { | ||
| const merged = { |
There was a problem hiding this comment.
Since this is gateway-specific logic, ideally there would be a way to solve this on the Gateway server or provider side. Earlier in the flow, summing things before we get into this code.
Since this is just reporting, and I get that I think the issue is this ai-core code is the only place we have all of the data come together, can we report the gateway provider metadata for each chunk/image-response within that chunk/response and leave it up to the caller to sum or aggregate or look across it if they choose to do so?
There was a problem hiding this comment.
The whole if (providerName === 'gateway') {} block is already gateway specific and needs to be refactored.
There was a problem hiding this comment.
Agreed that the whole block needs to be refactored. As a temporary solution to ship image models asap, 2 paths I see are:
- Go with my code in an earlier commit that sums the cost and sets the cost field to the true total cost
- Go with the current code in the PR that adds an
allCallsarray to the provider metadata containing the metadata for each request, and let the user sum the costs if they want. The only real issue with this is the metadata output gets quite long, and can be somewhat confusing because the data outside theallCallsarray is just for the last request and shows the cost of one image (same issue), so if a user doesn’t read into theallCallsarray they won’t realize what the actual cost is.
I was leaning towards option 1, but I realized that other fields other than cost are also being overwritten with the last request's metadata. This could be an issue as we'd only return fields like modelAttempts, generationId, resolvedProvider, etc for the last request if n > 1. So maybe option 2 is the best because it returns metadata from all calls. I'm just concerned about the duplicative-ness of the last call's metadata being outside allCalls (in the top level providerMetadata object) and also in the last position of the allCalls array.
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
|
Can we arrange things so that on ai side n is always 1 -- didn't we do something similar for embeddings? |
Pull request was closed
gr2m
added
ai/provider
3 participants
Background
I discovered a bug in the cost reporting for images generated via Gateway. If n > 1, the cost in the provider metadata would be reported as if n = 1 (but the correct amount would be debited). This is because the cost fields in the provider metadata were being merged without being summed across requests (since maxImagesPerCall = 1)
Summary
Merged with summation
Manual Verification
Ran Gateway requests and cost is now reported accurately
Checklist
pnpm changesetin the project root)