Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 118 additions & 0 deletions apps/docs/components/self-host-cards.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
"use client";

import { ArrowRight, Cloud, Container, Layers, Ship } from "lucide-react";
import Link from "next/link";
import { usePostHog } from "posthog-js/react";

import type { ReactNode } from "react";

interface Option {
title: string;
description: string;
href: string;
icon: ReactNode;
}

interface Group {
label: string;
options: Option[];
}

const groups: Group[] = [
{
label: "Single host",
options: [
{
title: "Docker",
description:
"Run every service in one container. The fastest way to a working instance.",
href: "/self-host/docker",
icon: <Container className="size-5" />,
},
{
title: "Docker Compose",
description:
"Run each service in its own container for more control over scaling and config.",
href: "/self-host/docker-compose",
icon: <Layers className="size-5" />,
},
],
},
{
label: "Cloud",
options: [
{
title: "Kubernetes (Helm)",
description:
"Deploy the official Helm chart to any cluster — EKS, GKE, AKS, or your own.",
href: "/self-host/kubernetes",
icon: <Ship className="size-5" />,
},
{
title: "AWS",
description: "EKS, RDS for Postgres, ElastiCache, and Secrets Manager.",
href: "/self-host/aws",
icon: <Cloud className="size-5" />,
},
{
title: "Google Cloud",
description: "GKE, Cloud SQL, Memorystore, and Secret Manager.",
href: "/self-host/gcp",
icon: <Cloud className="size-5" />,
},
{
title: "Azure",
description:
"AKS, Azure Database for PostgreSQL, Azure Cache for Redis, and Key Vault.",
href: "/self-host/azure",
icon: <Cloud className="size-5" />,
},
],
},
];

export function SelfHostCards() {
const posthog = usePostHog();

return (
<div className="not-prose flex flex-col gap-8">
{groups.map((group) => (
<div key={group.label} className="flex flex-col gap-3">
<h3 className="text-xs font-semibold uppercase tracking-wide text-fd-muted-foreground">
{group.label}
</h3>
<div className="grid grid-cols-1 gap-3 sm:grid-cols-2 lg:grid-cols-3">
{group.options.map((option) => (
<Link
key={option.href}
href={option.href}
onClick={() => {
posthog.capture("docs_self_host_card_click", {
option: option.title,
href: option.href,
});
}}
className="group relative flex flex-col gap-3 rounded-xl border border-fd-border bg-fd-card p-4 shadow-sm transition-all duration-200 hover:-translate-y-0.5 hover:shadow-md hover:border-fd-primary/40"
>
<div className="flex items-center justify-between">
<div className="flex items-center gap-3">
<div className="rounded-lg bg-fd-primary/10 p-2 text-fd-primary transition-colors duration-200 group-hover:bg-fd-primary/20">
{option.icon}
</div>
<h4 className="text-sm font-semibold tracking-tight text-fd-foreground">
{option.title}
</h4>
</div>
<ArrowRight className="size-4 -translate-x-1 text-fd-muted-foreground opacity-0 transition-all duration-200 group-hover:translate-x-0 group-hover:opacity-100 group-hover:text-fd-primary" />
</div>
<p className="text-[13px] leading-relaxed text-fd-muted-foreground">
{option.description}
</p>
</Link>
))}
</div>
</div>
))}
</div>
);
}
195 changes: 0 additions & 195 deletions apps/docs/content/self-host.mdx

This file was deleted.

75 changes: 75 additions & 0 deletions apps/docs/content/self-host/aws.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
---
title: AWS
description: Deploy LLM Gateway on AWS with EKS, RDS for Postgres, ElastiCache, and Secrets Manager.
icon: Cloud
---

import { Callout } from "fumadocs-ui/components/callout";

<Callout type="info">
**Deploy in one command.** Our [Enterprise
plan](https://llmgateway.io/enterprise) includes Terraform modules that
provision the cluster, database, cache, networking, and secrets and deploy LLM
Gateway on AWS — no manual setup required. [Learn
more](https://llmgateway.io/enterprise).
</Callout>

This guide covers a production deployment of LLM Gateway on AWS using EKS for the application services and managed AWS services for the backing stores.

## Architecture

| Component | AWS service |
| ---------- | ---------------------------- |
| Compute | Amazon EKS (Kubernetes) |
| PostgreSQL | Amazon RDS for PostgreSQL |
| Redis | Amazon ElastiCache for Redis |
| Secrets | AWS Secrets Manager |
| Ingress | AWS Load Balancer Controller |

## What to configure

### 1. PostgreSQL — Amazon RDS

Create an RDS for PostgreSQL instance in a private subnet. Enable automated backups and Multi-AZ for high availability. Note the connection string for `DATABASE_URL`.

### 2. Redis — Amazon ElastiCache

Create an ElastiCache for Redis cluster in the same VPC. Use it for `REDIS_URL`. A single-node cluster is fine to start; enable replication for production.

### 3. Compute — Amazon EKS

Create an EKS cluster with a managed node group sized for your traffic. Install the [AWS Load Balancer Controller](https://kubernetes-sigs.github.io/aws-load-balancer-controller/) to expose the gateway through an Application Load Balancer.

### 4. Networking

Run RDS and ElastiCache in private subnets and allow inbound traffic only from the EKS node security group. Expose only the gateway (and optionally the UI) to the internet through the load balancer.

### 5. Secrets — AWS Secrets Manager

Store `AUTH_SECRET`, `GATEWAY_API_KEY_HASH_SECRET`, and your provider API keys in Secrets Manager. Sync them into the cluster with the [External Secrets Operator](https://external-secrets.io/) or the AWS Secrets and Configuration Provider (ASCP).

## Deploy the Helm chart

With the backing services in place, deploy LLM Gateway with the [Helm chart](/self-host/kubernetes), pointing it at your RDS and ElastiCache endpoints:

```bash
helm install llmgateway oci://ghcr.io/theopenco/charts/llmgateway -f values.yaml
```

```yaml
config:
DATABASE_URL: "postgres://user:password@your-rds-endpoint:5432/llmgateway"
REDIS_URL: "redis://your-elasticache-endpoint:6379"
AUTH_SECRET: "from-secrets-manager"
GATEWAY_API_KEY_HASH_SECRET: "from-secrets-manager"
```

See the [Kubernetes guide](/self-host/kubernetes) for the full set of configurable values and how to scale the gateway.

<Callout type="info">
**Prefer not to wire this up by hand?** The [Enterprise
plan](https://llmgateway.io/enterprise) ships Terraform modules that stand up
the entire AWS stack — EKS, RDS, ElastiCache, networking, and secrets — and
deploy LLM Gateway in one command. [Talk to
us](https://llmgateway.io/enterprise#contact).
</Callout>
Loading
Loading