Skip to content

feat: add /tokenize for chat completion on aws bedrock converse api #2404

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
hustxiayang wants to merge 3 commits into
base: main
Choose a base branch
from
Open

feat: add /tokenize for chat completion on aws bedrock converse api #2404

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
91b491d
feat: add tokenize for AWS Bedrock Converse
hustxiayang Jul 9, 2026
943374c
refactor-codes
hustxiayang Jul 22, 2026
5aae66a
Merge branch 'main' into tokenize-bedrock
hustxiayang Jul 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions internal/apischema/awsbedrock/awsbedrock.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -638,6 +638,22 @@ type TitanEmbeddingResponse struct {
InputTextTokenCount int `json:"inputTextTokenCount"`
}

// CountTokensConverseInput mirrors the Converse input for the Bedrock CountTokens API.
// https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_CountTokens.html
type CountTokensConverseInput struct {
Messages []*Message `json:"messages"`
Comment on lines +643 to +644

@aabchoo aabchoo Jul 22, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing omitempty?

System []*SystemContentBlock `json:"system,omitempty"`
ToolConfig *ToolConfiguration `json:"toolConfig,omitempty"`
}

// CountTokensConverseRequest is the request structure for the Bedrock CountTokens API using Converse-style input.
// https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_CountTokens.html
type CountTokensConverseRequest struct {
Input struct {
Converse *CountTokensConverseInput `json:"converse"`
} `json:"input"`
}

// CountTokensInvokeModelRequest is the request structure for the Bedrock CountTokens API using InvokeModel-style input.
// The body is a base64-encoded model-specific request body (e.g., Anthropic Messages format).
// https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_CountTokens.html
Expand Down
2 changes: 2 additions & 0 deletions internal/endpointspec/endpointspec.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -459,6 +459,8 @@ func (TokenizeEndpointSpec) GetTranslator(schema filterapi.VersionedAPISchema, m
return translator.NewTokenizeToGCPAnthropicTranslator(schema.Version, modelNameOverride), nil
case filterapi.APISchemaAWSAnthropic:
return translator.NewTokenizeToAWSAnthropicTranslator(schema.Version, modelNameOverride), nil
case filterapi.APISchemaAWSBedrock:
return translator.NewTokenizeToAWSBedrockTranslator(modelNameOverride), nil
default:
return nil, fmt.Errorf("unsupported API schema for tokenize endpoint: backend=%s", schema.Name)
}
Expand Down
1 change: 1 addition & 0 deletions internal/endpointspec/endpointspec_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -481,6 +481,7 @@ func TestTokenizeEndpointSpec_GetTranslator(t *testing.T) {
{Name: filterapi.APISchemaGCPVertexAI},
{Name: filterapi.APISchemaGCPAnthropic},
{Name: filterapi.APISchemaAWSAnthropic},
{Name: filterapi.APISchemaAWSBedrock},
}

for _, schema := range supported {
Expand Down
Loading