Document the aggregation semantics of the built-in autoscaling metrics

Review flagged that AutoscalingContext advertised total_queued_requests as
"Number of requests currently queued" while _get_queued_requests now aggregates
the merged handle timeseries with the deployment's aggregation_function, so min
or max reports the window trough or peak instead of the current depth.

Fix the docs rather than the aggregation. total_running_requests is
total_num_requests - total_queued_requests, and the note above it already flags
that non-additive functions make the difference approximate; that identity only
holds while both sides share the function. Aggregating queued with mean while the
total uses max would yield max_total - mean_queued, which approximates nothing
and can go negative.

So: state the shared window and function on the class, where it reaches the
rendered API page; drop "currently" from the field comments; name the coupling at
the subtraction so the two sides do not get aggregated differently by accident;
and stop _get_queued_requests describing itself as a sum, which it never was.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: John Taylor <john.taylor@anyscale.com>

Author: johntaylor-cell

python/ray/serve/_private/autoscaling_state.py

@@ -652,7 +652,8 @@ def _get_queued_requests(self) -> float:
         """Calculate the total number of queued requests across all handles.
 
         Returns:
-            Sum of queued requests at all handles, aggregated from handle timeseries.
+            The merged instantaneous total of every handle's queued-requests
+            timeseries, aggregated over the window by `aggregation_function`.
         """
         return self._merge_and_aggregate_timeseries(
             self._collect_handle_queued_requests()

python/ray/serve/config.py

@@ -61,6 +61,10 @@ class AutoscalingContext:
     Note: The aggregated_metrics and raw_metrics fields support lazy evaluation.
     You can pass callables that will be evaluated only when accessed, with results
     cached for subsequent accesses.
+
+    Note: total_num_requests and total_queued_requests are both aggregated over
+    `look_back_period_s` using the deployment's `aggregation_function`, so under `min`
+    or `max` they report the window trough or peak rather than the current value.
     """
 
     def __init__(
@@ -112,10 +116,10 @@ def __init__(
 
         # Built-in metrics
         self._total_num_requests_value = (
-            total_num_requests  #: Total number of requests across all replicas.
+            total_num_requests  #: Ongoing (running + queued) requests.
         )
         self._total_queued_requests_value = (
-            total_queued_requests  #: Number of requests currently queued.
+            total_queued_requests  #: Requests queued at handles.
         )
 
         # Custom metrics - store potentially lazy callables privately
@@ -172,8 +176,9 @@ def total_queued_requests(self) -> float:
 
     @property
     def total_running_requests(self) -> float:
-        # NOTE: for non-additive aggregation functions, total_running_requests is not
-        # accurate, consider this is an approximation.
+        # Exact only because both sides share the deployment's `aggregation_function`;
+        # for non-additive ones (min/max) this difference is an approximation. Keep them
+        # on the same function -- aggregating either side differently breaks it outright.
         return self.total_num_requests - self.total_queued_requests
 
     @property