Feature: Implement Kiro AI Provider (AWS CodeWhisperer)

[ryuknull]

Feature: Implement Kiro AI Provider

Category: Feature
Priority: High
Related: Providers module (crates/infrastructure/providers-kiro)

---

What

Implement a Kiro AI provider that integrates with AWS CodeWhisperer / AWS Bedrock API. This enables Cortex users to use Kiro AI through the gateway.

---

Why

Kiro AI is a popular IDE coding assistant that uses AWS CodeWhisperer under the hood. Users want to route Kiro AI traffic through Cortex for:

• Unified API access across multiple providers
• Cost tracking and analytics
• Fallback and retry logic
• Protocol translation (OpenAI-compatible API)

---

API Overview

Based on OmniRoute's implementation, Kiro uses AWS CodeWhisperer streaming API:

Endpoint

POST /prod/codewhisperer/streaming

Base URL

https://us-east-1.api.aws

Authentication

• Bearer token (AWS credentials: access token)
• Header: Authorization: Bearer <token>

Additional Headers

Amz-Sdk-Request: attempt=1; max=3
Amz-Sdk-Invocation-Id: <uuid>
x-amzn-bedrock-cache-control: enable
anthropic-beta: prompt-caching-2024-07-31

---

Request Format

Kiro expects a simplified JSON payload (NOT OpenAI format):

{
  "conversationState": {
    "history": [
      {
        "userInputMessage": {
          "content": "message content",
          "modelId": "anthropic.claude-3-5-sonnet-20241022-v2:0",
          "origin": "AI_EDITOR",
          "userInputMessageContext": {
            "toolResults": [...],
            "tools": [...]
          }
        }
      },
      {
        "assistantResponseMessage": {
          "content": "response content",
          "toolUses": [...]
        }
      }
    ],
    "currentMessage": {
      "userInputMessage": {
        "content": "latest user message",
        "modelId": "anthropic.claude-3-5-sonnet-20241022-v2:0",
        "origin": "AI_EDITOR"
      }
    }
  },
  "profileArn": "arn:aws:codewhisperer:us-east-1:...",
  "inferenceConfig": {
    "maxTokens": 2048,
    "temperature": 0.5
  }
}

Message Conversion Rules

1. Roles: OpenAI system, tool, user → Kiro userInputMessage
2. Roles: OpenAI assistant → Kiro assistantResponseMessage
3. Tool calls: Convert to Kiro toolUses format
4. Images: Convert to { format: string, source: { bytes: string } }

Tool Schema Restrictions

Kiro API is strict:

• required arrays cannot be empty []
• additionalProperties is not supported
• Tool descriptions > 10,000 chars cause errors

---

Response Format (EventStream)

Kiro uses AWS EventStream binary format. Each frame has:

• 4-byte length header (big-endian)
• JSON payload with headers and payload

Event Types to Handle

1. assistantResponseEvent: Main text content

   { "content": "text response" }

2. codeEvent: Code blocks

   { "content": "code content" }

3. toolUseEvent: Tool call requests

   {
     "toolUseId": "...",
     "name": "tool_name",
     "input": { ... }
   }

4. messageStopEvent: End of response

5. contextUsageEvent: Context window usage

   { "contextUsagePercentage": 45.2 }

6. metricsEvent: Token usage

   {
     "inputTokens": 1000,
     "outputTokens": 500,
     "cacheReadTokens": 200,
     "cacheCreationTokens": 100
   }

7. meteringEvent: Usage tracking

---

Output Format (OpenAI SSE)

Convert Kiro events to OpenAI streaming format:

{
  "id": "chatcmpl-xxx",
  "object": "chat.completion.chunk",
  "created": 1234567890,
  "model": "kiro",
  "choices": [{
    "index": 0,
    "delta": { "role": "assistant", "content": "..." },
    "finish_reason": null
  }]
}

Tool Calls Format

{
  "delta": {
    "tool_calls": [{
      "index": 0,
      "id": "call_xxx",
      "type": "function",
      "function": {
        "name": "tool_name",
        "arguments": "{\"arg\": \"value\"}"
      }
    }]
  }
}

Finish Chunk

{
  "choices": [{
    "delta": {},
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 1000,
    "completion_tokens": 500,
    "total_tokens": 1500,
    "cache_read_input_tokens": 200,
    "cache_creation_input_tokens": 100
  }
}

---

What's Already There (Template)

Based on other providers in crates/infrastructure/providers-*/:

• KiroProviderConfig struct with:

  • id: ProviderId
  • access_token: String (Bearer token)
  • profile_arn: String
  • base_url: String (default: https://us-east-1.api.aws)
  • models: Vec<ModelId>
  • timeout_secs: u64

• KiroProvider struct with Client

• ProviderPort implementation

---

Implementation Tasks

1. Create Provider Module

crates/infrastructure/providers-kiro/
├── Cargo.toml
├── src/
│   ├── lib.rs          # Config + Provider struct
│   ├── client.rs       # HTTP client + request building
│   ├── translator.rs   # OpenAI → Kiro request translation
│   ├── parser.rs       # EventStream binary parsing
│   └── response.rs     # Kiro → OpenAI response translation
└── tests/
    └── integration.rs

2. Implement Request Translation

Convert CompletionRequest → Kiro format:

• convert_messages(): OpenAI messages → Kiro conversation state
• convert_tools(): OpenAI tools → Kiro tool specifications
• convert_images(): OpenAI image URLs → Kiro base64 format
• Handle role normalization and message merging

3. Implement EventStream Parser

Parse AWS EventStream binary frames:

• Read 4-byte length header
• Parse headers (:event-type, etc.)
• Extract payload JSON
• Handle different event types

4. Implement Response Translation

Convert Kiro events → OpenAI SSE:

• assistantResponseEvent → text delta
• codeEvent → code delta
• toolUseEvent → tool call delta
• messageStopEvent → finish chunk
• metricsEvent → usage data

5. Implement complete() and stream()

async fn complete(&self, req: &CompletionRequest) -> NuxaResult<CompletionResponse>
async fn stream(&self, req: &CompletionRequest) -> NuxaResult<BoxStream<'_, NuxaResult<StreamChunk>>>

6. Error Handling

• Handle AWS EventStream parsing errors
• Handle invalid credentials
• Handle rate limiting
• Handle model not available (HTTP 400)
• Handle timeout

7. Testing

• Unit tests for request translation
• Unit tests for response parsing
• Integration tests with mock AWS responses
• Test tool calls end-to-end
• Test streaming and non-streaming

---

Acceptance Criteria

• Kiro provider can complete chat completion requests
• Kiro provider can stream responses
• Tool calls work correctly
• Images are handled properly
• Token usage is tracked
• Error handling for invalid credentials
• Error handling for API errors (400, 429, 500)
• Tests pass

---

References

• OmniRoute Kiro implementation: https://github.com/diegosouzapw/OmniRoute
  • Executors: open-sse/executors/kiro.ts
  • Request translator: open-sse/translator/request/openai-to-kiro.ts
  • Response translator: open-sse/translator/response/kiro-to-openai.ts
  • MITM config: src/mitm/targets/kiro.ts
• AWS CodeWhisperer API docs (requires AWS account)

---

Notes

• Kiro is essentially AWS CodeWhisperer rebranded
• Uses AWS EventStream binary format (different from SSE)
• Requires valid AWS credentials with CodeWhisperer access
• Profile ARN identifies the CodeWhisperer profile to use
• Supports prompt caching with anthropic-beta header

[linear-code]

DALLAY-392