Skip to content

feat(anthropic): add support for Genkit v2 streaming API#342

Merged
xavidop merged 2 commits into
BloomLabsInc:mainfrom
planetarium:feat/genkit-v2-api-support
Sep 8, 2025
Merged

feat(anthropic): add support for Genkit v2 streaming API#342
xavidop merged 2 commits into
BloomLabsInc:mainfrom
planetarium:feat/genkit-v2-api-support

Conversation

@longfin

@longfin longfin commented Sep 5, 2025

Copy link
Copy Markdown
Contributor

Add Support for Genkit v2 Streaming API

Summary

This PR updates the Anthropic plugin to support the new Genkit v2 streaming API, which introduces a more structured approach to handling streaming requests and abort signals.

Changes

API Updates

  • Updated streaming callback interface: Migrated from a simple callback function to the new v2 API format with streamingRequested, sendChunk, and abortSignal parameters
  • Added apiVersion: 'v2': Specified the API version in the model definition for compatibility with Genkit v2
  • Integrated AbortSignal support: Properly pass AbortSignal to Anthropic SDK client calls to enable request cancellation

Code Improvements

  • Fixed indentation inconsistencies in toAnthropicToolResponseContent and fromAnthropicContentBlockChunk functions for better code readability
  • Updated all relevant test cases to reflect the new API structure

Breaking Changes

⚠️ BREAKING CHANGE: This update changes the streaming callback interface to align with Genkit v2 API. Projects using the old streaming callback format will need to update their code to use the new interface structure.

Migration Guide

Before (v1 API):

await model.generate(request, (chunk) => {
  // Handle chunk
});

After (v2 API):

await model.generate(request, {
  streamingRequested: true,
  sendChunk: (chunk) => {
    // Handle chunk
  },
  abortSignal: controller.signal
});

Testing

  • ✅ All existing tests pass
  • ✅ Test coverage remains at 95.76%
  • ✅ Updated test cases to verify new API functionality

Checklist

  • Code changes follow the project's coding standards
  • Tests have been updated/added
  • All tests pass locally
  • Documentation has been updated (if applicable)
  • Breaking changes are clearly documented

Related Issues

This change is part of the broader ecosystem update to support Genkit v2 API across all plugins.

Additional Context

This update ensures the Anthropic plugin remains compatible with the latest Genkit framework version while providing better support for request cancellation through AbortSignal integration.

@xavidop xavidop added the bump:minor Attach to PR to automatically bump minor version on merge label Sep 5, 2025
@xavidop

xavidop commented Sep 5, 2025

Copy link
Copy Markdown
Collaborator

Builds are failing

@longfin
longfin force-pushed the feat/genkit-v2-api-support branch from 71f90b9 to e775baa Compare September 5, 2025 18:36
@longfin
longfin marked this pull request as draft September 5, 2025 18:54
- Update streaming callback interface to use v2 API format with streamingRequested, sendChunk, and abortSignal
- Add apiVersion: 'v2' to model definition for compatibility
- Pass AbortSignal to Anthropic SDK client calls for proper request cancellation
- Update tests to reflect the new API changes
- Fix code indentation for better consistency

BREAKING CHANGE: This updates the plugin to work with Genkit v2 API which changes the streaming callback interface
@longfin
longfin force-pushed the feat/genkit-v2-api-support branch from e775baa to d68c288 Compare September 5, 2025 19:11
@longfin
longfin marked this pull request as ready for review September 5, 2025 19:11
@longfin

longfin commented Sep 5, 2025

Copy link
Copy Markdown
Contributor Author

@xavidop PTAL 🙏

@xavidop

xavidop commented Sep 6, 2025

Copy link
Copy Markdown
Collaborator

it is still failing

@longfin
longfin marked this pull request as draft September 8, 2025 01:26
@longfin
longfin force-pushed the feat/genkit-v2-api-support branch from e37ea1c to f831182 Compare September 8, 2025 02:16
@longfin

longfin commented Sep 8, 2025

Copy link
Copy Markdown
Contributor Author

Fixed build error (f831182).

it seems that there is no @nx/nx-linux-x64-gpu, but is @nx/nx-linux-x64-gnu. 😅

@longfin
longfin marked this pull request as ready for review September 8, 2025 02:19
@xavidop
xavidop merged commit 687408c into BloomLabsInc:main Sep 8, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bump:minor Attach to PR to automatically bump minor version on merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants