Skip to content

Commit 5b7e7c2

Browse files
aayush-kapooraayush-kapoor
authored
fix(google-vertex): don't send streamFunctionCallArguments for vertex doGenerate (#14352)
## Background #14314 introduced in #13929, we added support for streaming tool call arguments for the vertex provider. but the check for passing that parameter seeped through for doGenerate as well, producing an API error (the generate endpoint rejects that option) ## Summary - add a check to only pass `streamFunctionCallArguments` for doStream ## Manual Verification verified by running the example `examples/ai-functions/src/generate-text/google/vertex-grounding.ts` before and after the fix. before, it gave the api error. after: it resolves and produces an answer ## Checklist - [x] Tests have been added / updated (for bug fixes / features) - [x] Documentation has been added / updated (for bug fixes / features) - [x] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] I have reviewed this pull request (self-review) ## Related Issues fixes #14314
1 parent 48f65f5 commit 5b7e7c2

File tree

6 files changed

+181
-35
lines changed

6 files changed

+181
-35
lines changed

‎.changeset/six-pumas-refuse.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"@ai-sdk/google": patch
3+
---
4+
5+
fix(google-vertex): don't send streamFunctionCallArguments for unary API calls and change default to false

‎content/providers/01-ai-sdk-providers/16-google-vertex.mdx‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -350,7 +350,7 @@ The following optional provider options are available for Google Vertex models:
350350
Optional. When set to true, function call arguments will be streamed
351351
incrementally in streaming responses. This enables `tool-input-delta` events
352352
to arrive as the model generates function call arguments, reducing perceived
353-
latency for tool calls. Defaults to `true` for Vertex AI providers. Only supported on the Vertex AI API (not the Gemini API).
353+
latency for tool calls. Defaults to `false`. Only supported on the Vertex AI API (not the Gemini API) with Gemini 3+ models.
354354

355355
Consult [Google's Documentation](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling#streaming-fc) for details.
356356

@@ -469,8 +469,7 @@ For Gemini 3 Pro and later models on Vertex AI, you can stream function call
469469
arguments as they are generated by setting `streamFunctionCallArguments` to
470470
`true`. This reduces perceived latency when functions need to be called, as
471471
`tool-input-delta` events arrive incrementally instead of waiting for the
472-
complete arguments. This option is `true` by default and you can opt out by
473-
setting it to false.
472+
complete arguments. This option defaults to `false`.
474473

475474
```ts
476475
import { vertex } from '@ai-sdk/google-vertex';
@@ -491,7 +490,7 @@ const result = streamText({
491490
},
492491
providerOptions: {
493492
vertex: {
494-
streamFunctionCallArguments: false,
493+
streamFunctionCallArguments: true,
495494
} satisfies GoogleLanguageModelOptions,
496495
},
497496
});

‎examples/ai-functions/src/stream-text/google/vertex-stream-function-call-args.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,11 @@ run(async () => {
1616
}),
1717
},
1818
},
19+
providerOptions: {
20+
vertex: {
21+
streamFunctionCallArguments: true,
22+
},
23+
},
1924
includeRawChunks: true,
2025
});
2126

‎packages/google/src/google-generative-ai-language-model.test.ts‎

Lines changed: 138 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -5713,7 +5713,7 @@ describe('doStream', () => {
57135713
).toMatchInlineSnapshot(`undefined`);
57145714
});
57155715

5716-
it('should default streamFunctionCallArguments to true for Vertex provider even without provider option', async () => {
5716+
it('should default streamFunctionCallArguments to false for Vertex provider without provider option', async () => {
57175717
server.urls[TEST_URL_GEMINI_PRO].response = {
57185718
type: 'stream-chunks',
57195719
chunks: [
@@ -5759,14 +5759,10 @@ describe('doStream', () => {
57595759
],
57605760
});
57615761

5762-
expect((await server.calls[0].requestBodyJson).toolConfig)
5763-
.toMatchInlineSnapshot(`
5764-
{
5765-
"functionCallingConfig": {
5766-
"streamFunctionCallArguments": true,
5767-
},
5768-
}
5769-
`);
5762+
expect(
5763+
(await server.calls[0].requestBodyJson).toolConfig?.functionCallingConfig
5764+
?.streamFunctionCallArguments,
5765+
).toMatchInlineSnapshot(`undefined`);
57705766
});
57715767

57725768
it('should allow Vertex provider to opt out of streamFunctionCallArguments by setting it to false', async () => {
@@ -5826,6 +5822,139 @@ describe('doStream', () => {
58265822
).toBeUndefined();
58275823
});
58285824

5825+
it('should not send streamFunctionCallArguments for Vertex provider doGenerate (unary API)', async () => {
5826+
server.urls[TEST_URL_GEMINI_PRO].response = {
5827+
type: 'json-value',
5828+
body: {
5829+
candidates: [
5830+
{
5831+
content: { parts: [{ text: 'Hello' }], role: 'model' },
5832+
finishReason: 'STOP',
5833+
safetyRatings: SAFETY_RATINGS,
5834+
},
5835+
],
5836+
usageMetadata: {
5837+
promptTokenCount: 1,
5838+
candidatesTokenCount: 1,
5839+
totalTokenCount: 2,
5840+
},
5841+
},
5842+
};
5843+
5844+
const vertexModel = new GoogleGenerativeAILanguageModel('gemini-pro', {
5845+
provider: 'google.vertex.chat',
5846+
baseURL: 'https://generativelanguage.googleapis.com/v1beta',
5847+
headers: { 'x-goog-api-key': 'test-api-key' },
5848+
generateId: () => 'test-id',
5849+
});
5850+
5851+
await vertexModel.doGenerate({
5852+
prompt: TEST_PROMPT,
5853+
tools: [
5854+
{
5855+
type: 'function',
5856+
name: 'test-tool',
5857+
inputSchema: {
5858+
type: 'object',
5859+
properties: { value: { type: 'string' } },
5860+
required: ['value'],
5861+
additionalProperties: false,
5862+
$schema: 'http://json-schema.org/draft-07/schema#',
5863+
},
5864+
},
5865+
],
5866+
});
5867+
5868+
expect(await server.calls[0].requestBodyJson).toMatchInlineSnapshot(`
5869+
{
5870+
"contents": [
5871+
{
5872+
"parts": [
5873+
{
5874+
"text": "Hello",
5875+
},
5876+
],
5877+
"role": "user",
5878+
},
5879+
],
5880+
"generationConfig": {},
5881+
"tools": [
5882+
{
5883+
"functionDeclarations": [
5884+
{
5885+
"description": "",
5886+
"name": "test-tool",
5887+
"parameters": {
5888+
"properties": {
5889+
"value": {
5890+
"type": "string",
5891+
},
5892+
},
5893+
"required": [
5894+
"value",
5895+
],
5896+
"type": "object",
5897+
},
5898+
},
5899+
],
5900+
},
5901+
],
5902+
}
5903+
`);
5904+
});
5905+
5906+
it('should not send streamFunctionCallArguments for Vertex provider doGenerate even when explicitly set', async () => {
5907+
server.urls[TEST_URL_GEMINI_PRO].response = {
5908+
type: 'json-value',
5909+
body: {
5910+
candidates: [
5911+
{
5912+
content: { parts: [{ text: 'Hello' }], role: 'model' },
5913+
finishReason: 'STOP',
5914+
safetyRatings: SAFETY_RATINGS,
5915+
},
5916+
],
5917+
usageMetadata: {
5918+
promptTokenCount: 1,
5919+
candidatesTokenCount: 1,
5920+
totalTokenCount: 2,
5921+
},
5922+
},
5923+
};
5924+
5925+
const vertexModel = new GoogleGenerativeAILanguageModel('gemini-pro', {
5926+
provider: 'google.vertex.chat',
5927+
baseURL: 'https://generativelanguage.googleapis.com/v1beta',
5928+
headers: { 'x-goog-api-key': 'test-api-key' },
5929+
generateId: () => 'test-id',
5930+
});
5931+
5932+
await vertexModel.doGenerate({
5933+
prompt: TEST_PROMPT,
5934+
providerOptions: {
5935+
vertex: {
5936+
streamFunctionCallArguments: true,
5937+
},
5938+
},
5939+
});
5940+
5941+
expect(await server.calls[0].requestBodyJson).toMatchInlineSnapshot(`
5942+
{
5943+
"contents": [
5944+
{
5945+
"parts": [
5946+
{
5947+
"text": "Hello",
5948+
},
5949+
],
5950+
"role": "user",
5951+
},
5952+
],
5953+
"generationConfig": {},
5954+
}
5955+
`);
5956+
});
5957+
58295958
it('should only pass valid provider options', async () => {
58305959
prepareChunksFixtureResponse('google-text');
58315960

‎packages/google/src/google-generative-ai-language-model.ts‎

Lines changed: 27 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -88,22 +88,25 @@ export class GoogleGenerativeAILanguageModel implements LanguageModelV4 {
8888
return this.config.supportedUrls?.() ?? {};
8989
}
9090

91-
private async getArgs({
92-
prompt,
93-
maxOutputTokens,
94-
temperature,
95-
topP,
96-
topK,
97-
frequencyPenalty,
98-
presencePenalty,
99-
stopSequences,
100-
responseFormat,
101-
seed,
102-
tools,
103-
toolChoice,
104-
reasoning,
105-
providerOptions,
106-
}: LanguageModelV4CallOptions) {
91+
private async getArgs(
92+
{
93+
prompt,
94+
maxOutputTokens,
95+
temperature,
96+
topP,
97+
topK,
98+
frequencyPenalty,
99+
presencePenalty,
100+
stopSequences,
101+
responseFormat,
102+
seed,
103+
tools,
104+
toolChoice,
105+
reasoning,
106+
providerOptions,
107+
}: LanguageModelV4CallOptions,
108+
{ isStreaming = false }: { isStreaming?: boolean } = {},
109+
) {
107110
const warnings: SharedV4Warning[] = [];
108111

109112
const providerOptionsName = this.config.provider.includes('vertex')
@@ -184,9 +187,10 @@ export class GoogleGenerativeAILanguageModel implements LanguageModelV4 {
184187
? { ...resolvedThinking, ...googleOptions?.thinkingConfig }
185188
: undefined;
186189

187-
const streamFunctionCallArguments = isVertexProvider
188-
? (googleOptions?.streamFunctionCallArguments ?? true)
189-
: undefined;
190+
const streamFunctionCallArguments =
191+
isStreaming && isVertexProvider
192+
? (googleOptions?.streamFunctionCallArguments ?? false)
193+
: undefined;
190194

191195
const toolConfig =
192196
googleToolConfig ||
@@ -478,7 +482,10 @@ export class GoogleGenerativeAILanguageModel implements LanguageModelV4 {
478482
async doStream(
479483
options: LanguageModelV4CallOptions,
480484
): Promise<LanguageModelV4StreamResult> {
481-
const { args, warnings, providerOptionsName } = await this.getArgs(options);
485+
const { args, warnings, providerOptionsName } = await this.getArgs(
486+
options,
487+
{ isStreaming: true },
488+
);
482489

483490
const headers = combineHeaders(
484491
await resolve(this.config.headers),

‎packages/google/src/google-generative-ai-options.ts‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -192,9 +192,10 @@ export const googleLanguageModelOptions = lazySchema(() =>
192192
/**
193193
* Optional. When set to true, function call arguments will be streamed
194194
* incrementally via partialArgs in streaming responses. Only supported
195-
* on the Vertex AI API (not the Gemini API).
195+
* on the Vertex AI API (not the Gemini API) and only for Gemini 3+
196+
* models.
196197
*
197-
* @default true
198+
* @default false
198199
*
199200
* https://docs.cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling#streaming-fc
200201
*/

0 commit comments

Comments
 (0)