Prefix-cache hit-rate query uses different counter names than stable vLLM

[eikopf]

Summary

The V2 saturation analyzer's prefix-cache hit-rate query references vllm:prefix_cache_hits / vllm:prefix_cache_queries, but current vLLM emits these counters with the standard _total suffix. The query matches no series, so PrefixCacheHitRate silently reads as zero and the capacity model's input-token estimate loses the prefix-cache discount.

Evidence

Collector template (present at the v0.9.0 tag and on main), internal/collector/registration/saturation.go:

max by (instance, pod, llm_d_ai_variant) (rate(vllm:prefix_cache_hits{...}[5m]) / rate(vllm:prefix_cache_queries{...}[5m]))

vLLM 0.25.0 exposition (queried directly against a model-server pod's /metrics):

# TYPE vllm:prefix_cache_queries_total counter
# TYPE vllm:prefix_cache_hits_total counter
vllm:prefix_cache_queries_total{engine="0",model_name="..."} 5.12e+08
vllm:prefix_cache_hits_total{engine="0",model_name="..."} 4.57e+08

The repo's own benchmark config already uses the suffixed names (benchmark/config/templates/values/defaults.yaml):

• vllm:prefix_cache_hits_total
• vllm:external_prefix_cache_hits_total

so the collector template appears internally inconsistent with the rest of the repo rather than targeting an older vLLM on purpose.

Impact

No error is surfaced: domain/saturation_analyzer.go documents PrefixCacheHitRate as "Zero when prefix caching is disabled or metrics are unavailable", so deployments with prefix caching enabled silently get the metrics-unavailable behavior. On workloads with high prefix-hit ratios (ours observes ~0.89), the analyzer over-estimates input-token demand accordingly.

Suggested fix

Either query both spellings with an or fallback (the pattern already used for the target_model_name fallback in the flow-control queries), e.g.

rate(vllm:prefix_cache_hits_total{...}[5m]) or rate(vllm:prefix_cache_hits{...}[5m])

(and likewise for _queries), or handle the engine-specific naming the way the SGLang parser does. Happy to provide more environment details if useful.

Observed with: WVA v0.9.0 (also present on main @ bebbe88), vLLM 0.25.0, llm-d-router v0.9.0, metrics scraped via kube-prometheus-stack.

[YQ-Wang]

/assign

Hi, I'd like to work on this issue. I confirmed that the current saturation query uses the unsuffixed counters, while the repository's benchmark configuration uses the _total names.

I plan to update the vLLM prefix-cache query and add focused registration tests. Before implementing, could you confirm whether WVA should use the _total names exclusively or preserve compatibility with the unsuffixed names through an or fallback?

I lean toward _total only, but I'm happy to follow either direction.

[eikopf]

The unsuffixed counters are the actual Prometheus counter families, with the _total counter being the one we actually care about. vLLM is confusing here because it sometimes uses the unsuffixed name when it means the _total name; this works fine for them because the prometheus_client library they use normalizes the unsuffixed name and the _total name to the same counter, but the Prometheus client that WVA uses just passes the name unchanged.

TL;DR: just use the _total names, they're strictly more correct.