|
| 1 | +--- |
| 2 | +slug: ogx-bedrock-aws-auth |
| 3 | +title: "Use Amazon Bedrock with OGX Without Managing Bearer Tokens" |
| 4 | +authors: [skamenan7] |
| 5 | +tags: [ogx, aws, bedrock, sigv4, sts] |
| 6 | +date: 2026-06-02 |
| 7 | +--- |
| 8 | + |
| 9 | +OGX now signs Bedrock requests with standard AWS SigV4, so the server uses the same credential chain your platform already runs. No bearer tokens to manage, no custom auth plumbing in your application code. |
| 10 | + |
| 11 | +If your team uses IAM roles, IRSA, or STS for Bedrock access, this means OGX fits into your existing AWS identity model without extra moving parts. Apps talk to one OpenAI-compatible API while OGX handles the provider-specific auth behind the scenes. |
| 12 | + |
| 13 | +For the implementation details, see [issue #4730](https://github.com/ogx-ai/ogx/issues/4730) and [PR #5388](https://github.com/ogx-ai/ogx/pull/5388). |
| 14 | + |
| 15 | +<!--truncate--> |
| 16 | + |
| 17 | +## Why this helps teams adopt OGX |
| 18 | + |
| 19 | +If you are evaluating OGX for production use, this change makes Bedrock easier to fit into an existing AWS environment: |
| 20 | + |
| 21 | +- You can keep using existing OpenAI-compatible clients and agent frameworks. |
| 22 | +- You can use standard AWS identity flows, including IAM roles, IRSA, web identity, AWS profiles, and short-lived credentials. |
| 23 | +- You do not have to teach every app or service how Bedrock-specific auth works. |
| 24 | +- You can keep OGX as the abstraction layer, so switching between Bedrock and other providers stays an infrastructure choice instead of an application rewrite. |
| 25 | + |
| 26 | +The practical effect is less custom auth plumbing at the application layer. Teams can use Bedrock through OGX without giving up the AWS identity flows they already operate. |
| 27 | + |
| 28 | +## What shipped |
| 29 | + |
| 30 | +The core change is a SigV4 path for the Bedrock inference adapter. When `aws_bedrock_bearer_token` is absent, OGX now signs Bedrock requests with AWS SigV4 instead of assuming every request will carry a precomputed bearer token. |
| 31 | + |
| 32 | +That shipped with a few important pieces: |
| 33 | + |
| 34 | +- SigV4 request signing for the Bedrock OpenAI-compatible runtime. |
| 35 | +- STS web identity support through `aws_role_arn` and `aws_web_identity_token_file`. |
| 36 | +- Automatic refresh of temporary credentials. |
| 37 | +- Shared Bedrock config updates so the AWS auth story is consistent across the Bedrock provider code. |
| 38 | +- Compatibility with existing bearer-token mode, so this is additive rather than breaking. |
| 39 | + |
| 40 | +For users, the result is simple: if OGX has access to normal AWS credentials, it can talk to Bedrock without requiring a separate token-management workflow. |
| 41 | + |
| 42 | +## How it fits together |
| 43 | + |
| 44 | +From the application side, the client shape stays the same: point a standard client at OGX and let OGX handle the provider details. |
| 45 | + |
| 46 | +```python |
| 47 | +import requests |
| 48 | +from openai import OpenAI |
| 49 | + |
| 50 | +base_url = "http://localhost:8321" |
| 51 | +model_id = requests.get(f"{base_url}/v1/models", timeout=10).json()["data"][0]["id"] |
| 52 | + |
| 53 | +client = OpenAI(base_url=f"{base_url}/v1", api_key="ogx") |
| 54 | + |
| 55 | +response = client.chat.completions.create( |
| 56 | + model=model_id, |
| 57 | + messages=[{"role": "user", "content": "Explain why OGX is useful"}], |
| 58 | +) |
| 59 | +``` |
| 60 | + |
| 61 | +## Try it now |
| 62 | + |
| 63 | +If you already have AWS credentials available through a profile, IAM role, IRSA, or web identity, this is a short path to a working Bedrock request through OGX. |
| 64 | + |
| 65 | +### 1. Export your AWS environment |
| 66 | + |
| 67 | +```bash |
| 68 | +export AWS_DEFAULT_REGION=us-west-2 |
| 69 | + |
| 70 | +# Pick the option that matches your environment: |
| 71 | +# export AWS_PROFILE=default |
| 72 | +# export AWS_ROLE_ARN=arn:aws:iam::<account-id>:role/<role-name> |
| 73 | +# For EKS IRSA: |
| 74 | +# export AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token |
| 75 | +``` |
| 76 | + |
| 77 | +### 2. Write a minimal `config.yaml` |
| 78 | + |
| 79 | +```bash |
| 80 | +cat > config.yaml <<'EOF' |
| 81 | +version: 2 |
| 82 | +distro_name: bedrock-sigv4-demo |
| 83 | +apis: |
| 84 | + - inference |
| 85 | + - models |
| 86 | +providers: |
| 87 | + inference: |
| 88 | + - provider_id: bedrock-inference |
| 89 | + provider_type: remote::bedrock |
| 90 | + config: |
| 91 | + # aws_bedrock_bearer_token intentionally omitted so OGX uses the AWS credential chain |
| 92 | + region_name: ${env.AWS_DEFAULT_REGION:=us-west-2} |
| 93 | + aws_role_arn: ${env.AWS_ROLE_ARN:=} |
| 94 | + aws_web_identity_token_file: ${env.AWS_WEB_IDENTITY_TOKEN_FILE:=} |
| 95 | + aws_role_session_name: ${env.AWS_ROLE_SESSION_NAME:=ogx-bedrock-demo} |
| 96 | + session_ttl: ${env.AWS_SESSION_TTL:=3600} |
| 97 | +
|
| 98 | +storage: |
| 99 | + backends: |
| 100 | + kv_default: |
| 101 | + type: kv_sqlite |
| 102 | + db_path: ./.ogx/kvstore.db |
| 103 | + sql_default: |
| 104 | + type: sql_sqlite |
| 105 | + db_path: ./.ogx/sql_store.db |
| 106 | + stores: |
| 107 | + metadata: |
| 108 | + namespace: registry |
| 109 | + backend: kv_default |
| 110 | + inference: |
| 111 | + table_name: inference_store |
| 112 | + backend: sql_default |
| 113 | + max_write_queue_size: 10000 |
| 114 | + num_writers: 4 |
| 115 | + prompts: |
| 116 | + namespace: prompts |
| 117 | + backend: kv_default |
| 118 | +
|
| 119 | +registered_resources: |
| 120 | + models: |
| 121 | + - metadata: {} |
| 122 | + model_id: openai.gpt-oss-20b-1:0 |
| 123 | + provider_id: bedrock-inference |
| 124 | + provider_model_id: openai.gpt-oss-20b-1:0 |
| 125 | + model_type: llm |
| 126 | +EOF |
| 127 | +``` |
| 128 | + |
| 129 | +### 3. Start OGX |
| 130 | + |
| 131 | +```bash |
| 132 | +uv run ogx run --port 8321 ./config.yaml |
| 133 | +``` |
| 134 | + |
| 135 | +Leave that terminal running. In a second terminal, use the steps below. |
| 136 | + |
| 137 | +### 4. Verify the model is available |
| 138 | + |
| 139 | +```bash |
| 140 | +MODEL_ID=$(curl -s http://localhost:8321/v1/models | jq -r '.data[0].id // empty') |
| 141 | +test -n "$MODEL_ID" && echo "Using model: $MODEL_ID" |
| 142 | +``` |
| 143 | + |
| 144 | +If you want to inspect the full model list: |
| 145 | + |
| 146 | +```bash |
| 147 | +curl -s http://localhost:8321/v1/models | jq |
| 148 | +``` |
| 149 | + |
| 150 | +### 5. Send your first request with `curl` |
| 151 | + |
| 152 | +```bash |
| 153 | +curl -s -X POST "http://localhost:8321/v1/chat/completions" \ |
| 154 | + -H "Content-Type: application/json" \ |
| 155 | + -d "{ |
| 156 | + \"model\": \"$MODEL_ID\", |
| 157 | + \"messages\": [{\"role\": \"user\", \"content\": \"Which planet do humans live on?\"}], |
| 158 | + \"stream\": false |
| 159 | + }" | jq -r '.choices[0].message.content' |
| 160 | +``` |
| 161 | + |
| 162 | +### 6. Send the same request from the OpenAI Python client |
| 163 | + |
| 164 | +```bash |
| 165 | +uv run python - <<'PY' |
| 166 | +import requests |
| 167 | +from openai import OpenAI |
| 168 | +
|
| 169 | +base_url = "http://localhost:8321" |
| 170 | +model_id = requests.get(f"{base_url}/v1/models", timeout=10).json()["data"][0]["id"] |
| 171 | +
|
| 172 | +client = OpenAI(base_url=f"{base_url}/v1", api_key="ogx") |
| 173 | +
|
| 174 | +response = client.chat.completions.create( |
| 175 | + model=model_id, |
| 176 | + messages=[{"role": "user", "content": "Which planet do humans live on?"}], |
| 177 | + stream=False, |
| 178 | +) |
| 179 | +
|
| 180 | +print(response.choices[0].message.content) |
| 181 | +PY |
| 182 | +``` |
| 183 | + |
| 184 | +If you are only doing a quick local spike and already have a pre-signed Bedrock bearer token, that path still works. For long-running deployments, leave bearer auth unset and let OGX use the standard AWS identity flow instead. |
| 185 | + |
| 186 | +If you do use a bearer token, make sure it was generated for the same AWS Region that OGX is using. A token scoped to `us-east-2` will be rejected by a Bedrock endpoint configured for `us-west-2`, and vice versa. |
| 187 | + |
| 188 | +There are two common paths here: direct SigV4-backed inference for `chat/completions`, and a Bedrock-backed `Responses` path for higher-level workflows such as tool calling. The steps above focus on the shorter `chat/completions` path. |
| 189 | + |
| 190 | +That path has been exercised end to end through the standard OGX surface: model resolution from `/v1/models`, non-streaming and streaming `chat/completions`, fallback to SigV4 when empty bearer overrides are supplied, successful bearer-token overrides when explicitly provided, rejection of invalid bearer overrides, concurrent request isolation, and repeated request smoke checks. For someone evaluating OGX, that is the important part: the AWS-native path works through the same API shape your applications already use. |
| 191 | + |
| 192 | +## Migration |
| 193 | + |
| 194 | +If your config already uses `api_key` or `aws_bearer_token_bedrock`, it still works. No changes required. OGX accepts both the old and new field names through Pydantic aliases. |
| 195 | + |
| 196 | +For new deployments, use the canonical name `aws_bedrock_bearer_token` in your config and `AWS_BEDROCK_BEARER_TOKEN` as the environment variable. Per-request bearer overrides via the `x-ogx-provider-data` header also accept both old and new field names. |
0 commit comments