Skip to content

fix: warn when refresh-metrics-interval exceeds metricsStalenessThres… - #2645

Open
varad-ahirwadkar wants to merge 1 commit into
llm-d:mainfrom
varad-ahirwadkar:flowcontrol
Open

fix: warn when refresh-metrics-interval exceeds metricsStalenessThres…#2645
varad-ahirwadkar wants to merge 1 commit into
llm-d:mainfrom
varad-ahirwadkar:flowcontrol

Conversation

@varad-ahirwadkar

Copy link
Copy Markdown
Contributor

What type of PR is this?
/kind bug

What this PR does / why we need it:
When --refresh-metrics-interval exceeds a saturation detector's metricsStalenessThreshold, endpoint metrics may become stale between polls. This can cause the detector to treat healthy endpoints as saturated, leading to backpressure and increased request latency.

This PR adds a startup warning that identifies the detector and both effective values, helping operators detect and correct the configuration before serving traffic.

Which issue(s) this PR fixes:
Fixes #2514

Release note (write NONE if no user-facing change):

Added a startup warning when `--refresh-metrics-interval` exceeds a saturation detector's `metricsStalenessThreshold`.

@github-actions github-actions Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. kind/bug Categorizes issue or PR as related to a bug. area/epp area/flowcontrol labels Sep 1, 2026

@ahg-g ahg-g left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't recommend expanding and adding yet another interface, instead we can put this in the handle within a perhaps metadata struct that the plugin can check and do whatever it sees fit (fail creation, warn etc.).

@varad-ahirwadkar

Copy link
Copy Markdown
Contributor Author

Hi @ahg-g
Thanks for the suggestion! I'll rework this to expose the effective refresh interval via the Handle and move the comparison into the UtilizationDetectorFactory.

@github-actions github-actions Bot added area/dev Related to dev tooling, experience, etc. area/scheduling labels Sep 3, 2026
…hold

Signed-off-by: Varad Ahirwadkar <varad.ahirwadkar1@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/dev Related to dev tooling, experience, etc. area/epp area/flowcontrol area/scheduling kind/bug Categorizes issue or PR as related to a bug. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

flowcontrol: warn when metrics refresh interval exceeds staleness threshold

2 participants