llm-d
Repositories list
22 repositories
- Achieve state of the art inference performance with modern accelerators on Kubernetes
- llm-d benchmark scripts and tooling
llm-d-batch-gateway
Publicllm-d-router
Publicllm-d Router: The intelligent entry point for inference requestsllm-d.github.io
PublicWebsite for llm-d: This repository builds the website seen at llm-d.aillm-d-go-template
Public templatellm-d-prism
Public- A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.
llm-d-kv-cache
PublicDistributed KV cache scheduling & offloading librariesllm-d-async
Publicllm-d-infra
Publicllm-d-api-extensions
Publicllm-d-latency-predictor
Publicllm-d-pd-utils
Public.project
Publicllm-d-python-template
Public template.github
Publicllm-d-routing-sidecar
Public archiveIncubating P/D sidecar for llm-dllm-d-deployer
Public archiveHelm charts for llm-dllm-d-model-service
Public archiveSimplified model deployment on llm-d
ProTip! When viewing an organization's repositories, you can use the
props. filter to filter by custom property.