feat: add priority based holdback strategies in flow control

[loicmarchal]

What type of PR is this?
/kind feature

What this PR does / why we need it:
Add progressive holdback based on priorities.

It implements the priority-holdback-policy plugin (flowcontrol.UsageLimitPolicy), which computes differentiated admission ceilings per priority level. Lower-priority traffic is gated at lower saturation thresholds, reserving capacity for higher-priority work as the pool approaches saturation.

It replaces the default static usage limit policy's binary backpressure approach with priority-aware stepped gating.

Two gating strategies are supported:

• stepwise-spread: equal ceiling spacing by rank, ignoring numerical priority values.
  $$c_i = c_{max} - i \times \frac{c_{max} - c_{min}}{N - 1}$$
• linear-proportional: ceiling spacing proportional to numerical priority differences.
  $$c_i = c_{min} + r_i \times (c_{max} - c_{min})$$
  $$r_i = \frac{p_i - p_{min}}{p_{max} - p_{min}}$$

Both are the same linear mapping applied to different input domains (rank vs value). With evenly distributed priorities, they converge.

It's a first implementation for a v1. More strategies could be added to offer more control of the ceilings distribution to the user (e.g. Sigmoid curve instead of linear).

Which issue(s) this PR fixes:
Fixes #1185

Release note (write NONE if no user-facing change):

Add priority-aware stepped gating policy as an alternative to the default binary backpressure approach

[github-actions]

🚨 Unsigned commits detected! Please sign your commits.

For instructions on how to set up GPG/SSH signing and verify your commits, please see GitHub Documentation.

[LukeAVanDrie]

Didn't get around to reviewing this today; sorry @loicmarchal! Will take a look when I am back in office Tuesday. Let's target this for v0.10 if possible and try to land this alongside eviction (which @RishabhSaini has a draft for).

[loicmarchal]

Didn't get around to reviewing this today; sorry @loicmarchal! Will take a look when I am back in office Tuesday. Let's target this for v0.10 if possible and try to land this alongside eviction (which @RishabhSaini has a draft for).

Thank you @LukeAVanDrie. No worries. Sounds good for v0.10

[loicmarchal]

@LukeAVanDrie I implemented a new plugin in its own package. I hesitated between this approach and making use of the existing usage limit. The way the usage limit policy was implemented uses a function that is simply called in the ComputeLimit function. We could look into making the usage limit more generic and simply return the function associated with the strategy defined in the config. That might require some refactoring, and maybe adapting the doc a little bit. I'll take a look at it.

[loicmarchal]

I'd recommend two orthogonal fields instead:

I agree. That was a main argument in my comment

My proposal:

parameters:
    shape: linear      # optional, default: linear   (future: sigmoid) => clarify in the doc we currently only support linear
    domain: rank       # optional, default: rank     (rank | value)
    minCeiling: 0.4    # required
    maxCeiling: 0.9    # optional, default: 1.0

This way, we are future-proof for sigmoid, exponential, step, etc. At the same time, if we stay as it is, nothing will break.

If ok with you, I'll implement that change, then maybe we can merge and add sigmoid (or others) as a future implementation?

[loicmarchal]

Thank you for your review @LukeAVanDrie. I changed the config.

[LukeAVanDrie]

/lgtm

Thanks @loicmarchal!

[loicmarchal]

@ahg-g @liu-cong does it look ok for you? Can it be merged?

[lioraron]

@ahg-g @liu-cong @elevran @shmuelk This PR is critical for batch-gateway integration with flow-control - #1185 (comment).

Can you please look into merging this?

Thanks.