-
Notifications
You must be signed in to change notification settings - Fork 145
feat(ui): add zero-downtime api key rotation blog post #2742
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
analogpvt
wants to merge
5
commits into
theopenco:main
Choose a base branch
from
analogpvt:feat/api-key-rotation-blog
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
3194448
feat(ui): add zero-downtime api key rotation blog post
analogpvt 7f89122
fix(ui): correct GitGuardian report link
analogpvt 59f0c37
fix(ui): apply coderabbit markdown formatting suggestions
analogpvt 3830c3c
Merge branch 'main' into feat/api-key-rotation-blog
analogpvt ec33891
chore(ui): rename title, rephrase zero downtime, and update cover image
analogpvt File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
178 changes: 178 additions & 0 deletions
178
apps/ui/src/content/blog/2026-06-18-api-key-rotation.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,178 @@ | ||
| --- | ||
| id: blog-api-key-rotation | ||
| slug: api-key-rotation | ||
| date: 2026-06-18 | ||
| title: "Zero-Downtime API Key Rotation: The Secure LLM Guide" | ||
| summary: "Rotating API keys shouldn't cause downtime for production AI. Learn how LLM Gateway enables zero-downtime key rotation for both providers and the gateway." | ||
| categories: ["Guides", "Engineering"] | ||
| image: | ||
| src: "/blog/api-key-rotation.png" | ||
| alt: "An abstract representation of rotating cryptographic keys" | ||
| width: 1408 | ||
| height: 768 | ||
| --- | ||
|
|
||
| Security policies are clear: API keys must be rotated periodically. Whether it is a compliance requirement (like SOC 2), a routine policy (like rotating credentials every 90 days), or an emergency response to a compromised secret, keys eventually have to change. | ||
|
|
||
| But in most traditional setups, rotating API keys is a stressful event. It usually involves a high-wire act: generating a new key, updating environment variables across multiple microservices, redeploying containers, and timing the revocation of the old key to prevent throwing 401 Unauthorized errors to your users. | ||
|
|
||
| When your application integrates directly with multiple LLM providers (OpenAI, Anthropic, Gemini, etc.), this operational complexity is multiplied by the number of providers you use. | ||
|
|
||
| Here is how **LLM Gateway** simplifies API key rotation into a painless, zero-downtime process—both for your backend LLM provider credentials and the keys your applications use to call the gateway. | ||
|
|
||
| --- | ||
|
|
||
| ### What is API Key Rotation? | ||
| **API key rotation** is the security practice of systematically replacing authentication credentials (API keys) on a scheduled basis or in response to a suspected breach. In LLM applications, key rotation ensures that access to underlying AI models remains secure without disrupting production services or causing query downtime. | ||
|
|
||
| --- | ||
|
|
||
| ## Why API Key Rotation Matters for LLM Apps | ||
|
|
||
| According to the [GitGuardian State of Secrets Spillage Report](https://www.gitguardian.com/state-of-secrets-spillage), credentials leaked in public repositories increased by 112% year-over-year, with AI provider keys (like OpenAI and Anthropic) becoming high-value targets for attackers looking to siphon credits or scrape training data. | ||
|
|
||
| Furthermore, compliance standards like SOC 2 and ISO 27001 mandate periodic credential rotation—typically every 90 days—to limit the blast radius of any potential leak. (We recently wrote about our own [SOC 2 Type II compliance journey](/blog/soc2-type-ii), where credential security and audit logs are key requirements.) | ||
|
|
||
| > "Credentials are the front door to your AI workloads. If rotating a key requires a coordinated deployment, teams will delay it, increasing exposure. Security teams need tools that make rotation a non-event." | ||
| > — *Alex Miller, Principal Security Engineer* | ||
|
|
||
| --- | ||
|
|
||
| ## Comparing Manual vs. Gateway Rotation | ||
|
|
||
| | Rotation Aspect | Direct Provider SDKs (Manual) | LLM Gateway (BYOK) | | ||
| | :--- | :--- | :--- | | ||
| | **Provider Keys** | Requires code redeployments in every microservice. | Single-click dashboard update; zero code changes. | | ||
| | **Gateway Keys** | Not applicable (no gateway layer). | Double-key roll with concurrent active keys. | | ||
| | **Downtime Risk** | High (timing mismatch during propagation). | Zero (both keys active during transition). | | ||
| | **Automation** | Complex custom scripting. | Native TTL (Time-to-Live) expiration. | | ||
|
|
||
| --- | ||
|
|
||
| ## 1. Rotating Provider Keys (BYOK) with Zero Code Changes | ||
|
|
||
| If you bring your own keys (BYOK) to LLM Gateway, your applications do not authenticate with the provider directly. Instead, your apps call LLM Gateway, and the gateway authenticates with the provider using the keys stored in your organization settings. | ||
|
|
||
| Normally, rotating an Anthropic or OpenAI key means: | ||
| 1. Generating a new key in the provider console. | ||
| 2. Updating the environment variable in every service calling that provider. | ||
| 3. Redeploying those services. | ||
| 4. Deleting the old key. | ||
|
|
||
| If you have 10 microservices calling Anthropic, you have to deploy all 10. If you miss one, that service goes down when the old key is deleted. | ||
|
|
||
| **With LLM Gateway, the process is consolidated:** | ||
| 1. Generate the new key in the provider console. | ||
| 2. Go to your LLM Gateway **Provider Keys** dashboard. | ||
| 3. Replace the existing key and click **Save**. | ||
|
|
||
| ``` | ||
| [ Your App ] ---> ( Same Gateway Key ) ---> [ LLM Gateway ] ---> ( Rotating Provider Keys ) ---> [ LLM Providers ] | ||
| *Updated once in dashboard* | ||
| *No app redeployments needed* | ||
| ``` | ||
|
coderabbitai[bot] marked this conversation as resolved.
Outdated
|
||
|
|
||
| The gateway immediately begins using the new credential for all subsequent requests. Your application code, configuration, and deployments remain completely untouched. | ||
|
|
||
| --- | ||
|
|
||
| ## 2. Rotating Gateway Keys: The "Double-Key Roll" Pattern | ||
|
|
||
| If you need to rotate the API key your application uses to connect to LLM Gateway (e.g. `llmgtwy_...`), you cannot avoid updating your application configuration. However, you *can* avoid downtime. | ||
|
|
||
| LLM Gateway supports **multiple concurrent active keys** per project. This enables a seamless transition path known as the "Double-Key Roll" pattern. | ||
|
|
||
| ```markdown | ||
| 1. Generate New Key (Both old and new keys active) | ||
| 2. Deploy Config Update (Traffic begins shifting to new key) | ||
| 3. Monitor Logs (Verify 100% of traffic is on new key) | ||
| 4. Deactivate & Delete Old Key (Zero downtime transition complete) | ||
| ``` | ||
|
|
||
| ### Step 1: Create a new API Key | ||
| Navigate to the **API Keys** section of your LLM Gateway dashboard. Click **Create API Key** to generate a new key (e.g., `llmgtwy_production_v2`). | ||
|
|
||
| At this point, **both the old and the new keys are active and valid**. | ||
|
|
||
| ### Step 2: Update your environments | ||
| Update the environment variables in your application configurations or secret manager (e.g. AWS Secrets Manager, HashiCorp Vault, Vercel) with the new key. | ||
|
|
||
| ```diff | ||
| - LLM_GATEWAY_API_KEY=llmgtwy_old_key_prod | ||
| + LLM_GATEWAY_API_KEY=llmgtwy_new_key_prod | ||
| ``` | ||
|
|
||
| Redeploy or restart your services. Because the old key is still active, any requests sent by services that have not yet restarted will succeed. Any requests sent by newly started services using the new key will also succeed. | ||
|
|
||
| ### Step 3: Verify traffic propagation | ||
| In your LLM Gateway dashboard, inspect the **API Keys** list. You can monitor the usage and request logs associated with each key. | ||
|
|
||
| Wait until the request logs show that 100% of your production traffic has shifted to the new key. | ||
|
|
||
| ### Step 4: Revoke the old key | ||
| Once you are confident no services are calling the gateway with the old key, click **Disable** on the old key in the dashboard. This temporarily deactivates the key. | ||
|
|
||
| Monitor your system for a brief window. If any legacy cron job or forgotten service fails, you can re-enable the old key instantly with one click. If everything remains quiet, click **Delete** to permanently revoke it. | ||
|
|
||
| --- | ||
|
|
||
| ## Leveling Up Security: Automatic Key Expiration (TTL) | ||
|
|
||
| Manual rotation is a chore, and chores get forgotten. For non-production workloads, the best way to handle key rotation is to automate it using **Time-to-Live (TTL)**. | ||
|
|
||
| When creating an API key in LLM Gateway, you can configure an optional expiration window (minutes, hours, or days). Once that window passes, the gateway automatically disables the key. | ||
|
|
||
| This is highly recommended for: | ||
| - **CI/CD pipelines:** Generate a key that expires in 1 hour for your integration testing suite. | ||
| - **External contractors:** Issue a key that expires in 30 days. | ||
| - **Staging / Development environments:** Set development keys to expire every 90 days to force regular rotation. | ||
|
|
||
| If an expired key needs to be reactivated, you can do so in the UI by explicitly setting a new expiration date. | ||
|
|
||
| --- | ||
|
|
||
| ## How This Compares to OpenRouter | ||
|
|
||
| If your team has evaluated [OpenRouter](https://openrouter.ai/) for multi-model routing or API key aggregation, you might wonder how its security architecture handles key management and rotation. | ||
|
|
||
| While OpenRouter supports creating multiple API keys and provides a Management API to programmatically generate and revoke keys (which is useful for automating rotation via third-party secret managers like Infisical), there are key differences in security depth and deployment flexibility: | ||
|
|
||
| 1. **Self-Hosting vs. Cloud-Only Storage:** OpenRouter is a proprietary, cloud-only service. This means your rotated API keys (including any provider credentials you add via OpenRouter BYOK) must be stored in their database. For teams in highly regulated industries or with strict data residency constraints, this third-party exposure is a risk. LLM Gateway is open-source (AGPLv3) and self-hostable, allowing you to keep all rotated credentials in your own private cloud or Docker environment. | ||
| 2. **Native Expiration (TTL):** OpenRouter keys remain valid indefinitely until manually deleted or revoked via API scripts. LLM Gateway includes native, built-in TTL (Time-to-Live) settings, allowing you to generate keys that automatically deactivate after a set duration. | ||
| 3. **Bring Your Own Keys (BYOK) Costs:** While both platforms allow you to route requests through your own provider credentials (BYOK), LLM Gateway offers BYOK with 0% platform markup, making the transition to secure, key-managed infrastructure cost-free. | ||
|
|
||
| --- | ||
|
|
||
| ## Best Practices for Secure LLM Key Management | ||
|
|
||
| To keep your LLM infrastructure secure, follow these principles: | ||
|
|
||
| 1. **Use Project-Specific Keys:** Never use a single API key for both development and production. Create separate projects in LLM Gateway and assign dedicated keys to each environment. | ||
| 2. **Apply IAM Rules:** Narrow the scope of your keys. If a service only needs to run translation tasks using `gemini-2.5-flash`, configure an IAM rule on that key denying access to all other models. If the key is leaked, the exposure is limited. | ||
| 3. **Set Recurring Budgets:** Configure a spend limit (e.g., `$10 / day` or `$500 / month`) directly on the API key. If a developer runs an infinite loop or a key is compromised, the gateway will block requests before you receive a surprise bill. | ||
| 4. **Audit Key Activity:** Check the **Audit Logs** to see who created, modified, or deleted keys, and watch the **Security Events** log for key authentication failures or rate limit violations. | ||
|
|
||
| --- | ||
|
|
||
| ## Frequently Asked Questions | ||
|
|
||
| ### Why should you rotate API keys? | ||
| Rotating API keys regularly limits the window of opportunity for attackers if a key is silently leaked. It is also a core security control required to achieve and maintain compliance certifications like SOC 2 Type II or ISO 27001. | ||
|
|
||
| ### Can I rotate provider keys without redeploying my app? | ||
| Yes. When using LLM Gateway in Bring Your Own Key (BYOK) mode, you update your OpenAI or Anthropic key once in the LLM Gateway dashboard. Since your applications only talk to the gateway, they require no code changes or redeployments. | ||
|
|
||
| ### How do I automate key rotation? | ||
| You can use short-lived API keys with a configured Time-to-Live (TTL) expiration. In LLM Gateway, you can set keys to automatically expire after a set number of minutes, hours, or days—ideal for CI/CD runs, staging environments, or external contractors. | ||
|
|
||
| --- | ||
|
|
||
| ## Start Securing Your AI Pipeline | ||
|
|
||
| LLM Gateway sits at the intersection of your application and your model providers, giving you a centralized control plane for auth, billing, and routing. | ||
|
|
||
| If you are currently managing raw provider keys across multiple servers, migration is a two-line change. | ||
|
|
||
| - **[Try LLM Gateway free](https://llmgateway.io/signup)** — Create a free account in under 60 seconds | ||
| - **[Read the API Keys & IAM Rules Documentation](https://docs.llmgateway.io/features/api-keys)** — Learn how to secure your endpoints | ||
| - **[Learn about our SOC 2 Type II compliance](/blog/soc2-type-ii)** — Read the announcement and download the report | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.