[Roadmap] vLLM Production Stack 2026 Roadmap

[ruizhang0101]

Router

• (P0) XpYd disaggregated prefill support
• (P1) Routing to external providers like OpenAI or Anthropic (feature: [RFC] Enhance router to support external providers (OpenAI, Anthropic, commercial models) #883 [Feat] Implement OpenAI external provider #902 )
• (P1) Router-side request queuing support ([FEAT][RFC] Add router-side request queuing proposal #876 )
• (P2) Implement predictive routing based on future workloads
• (P2) Implement priority routing
• (P2) Improve router performance
  • Compile current router with Nuitka
  • Prototype Rust/Go/Nginx-based router frontend

Advanced Deployment

• (P0) Advanced autoscaling based on modern inference metrics
• (P0) Implement KV-cache–aware and prefix-aware routing logic in Gateway Inference extension and CRD ([Feat][Operator] Add prefixaware and kvaware routing options to VLLMRouter CRD #881 )
• (P0) Implement disaggregated prefill support in CRD ([Feat] Support PD Disaggregation via CRD Operator #841 )
• (P0) Integrate KEDA-based scaling in CRD ([Feat] Support KEDA Auto Scaling in Production Stack Operator  #903 )
• (P1) Make existing deployment methods (Helm/CRD/Gateway Inference) more Kubernetes-native
• (P2) Support automatic configuration of engine parameters

Support for Modern Workloads

• (P0) Support deployment with vLLM Omni
• (P1) Intelligent routing for agent workloads and multimodal models
• (P2) End-to-end deployment for multimodal models
• (P2) First-class support for RL workloads

Safety and Security

• (P0) Request migration when the vLLM instance fails
• (P1) Ship router extension config with pre-defined safety policies, rate limits, and logging enabled
• (P1) Add optional PII detection in the request path

CI/CD and Misc

• (P0) Release bot for automated releases (Helm chart, K8s controller packages, Docker images)
• (P2) GitHub Actions for building router Docker images for multiple architectures
• (P2) Tutorials covering more cloud platforms, models, and features

If any of the items you wanted are not on the roadmap, your suggestion and contribution are strongly welcomed! Please feel free to comment in this thread, open a feature request, or create an RFC.

Happy vLLMing!

[surak]

Forgive me the ignorance, but why there's a python vllm-router here and a completely different one in https://github.com/vllm-project/router ? This makes it confusing to choose in a production stack.

[ruizhang0101]

Forgive me the ignorance, but why there's a python vllm-router here and a completely different one in https://github.com/vllm-project/router ? This makes it confusing to choose in a production stack.

Hi, Thanks for raising this. That router is a new project with rust implementations which mainly focus on router part. This project is more of a complete solution including the router and the full stack deployment.

[ardecode]

@ruizhang0101 , Thanks for putting together this roadmap!
Are any of these items already being actively worked on, or is the community free to pick them up? I’m interested in contributing to the router area and would be happy to start drafting a design or PR if those areas are open.

Apologies if this information is already documented somewhere and I missed it.

[ruizhang0101]

@ruizhang0101 , Thanks for putting together this roadmap! Are any of these items already being actively worked on, or is the community free to pick them up? I’m interested in contributing to the router area and would be happy to start drafting a design or PR if those areas are open.

Apologies if this information is already documented somewhere and I missed it.

Hi! They are free for the community to pick up! I will update the item here accordingly when there is a RFC or PR or anyone that are related to the item here.

[ardecode]

@ruizhang0101 For Router-side request queuing support, I notice there’s already an earlier attempt in PR #626 (“Router-side queuing support”). Is the intention to revive/rebase and finish #626, or would you prefer a fresh PR (possibly borrowing design ideas from #626)?

[ruizhang0101]

@ruizhang0101 For Router-side request queuing support, I notice there’s already an earlier attempt in PR #626 (“Router-side queuing support”). Is the intention to revive/rebase and finish #626, or would you prefer a fresh PR (possibly borrowing design ideas from #626)?

I think that PR is in stale mode right now. We can start a fresh PR from there. :)))