KEDA Prometheus Trigger – HPA Receives ~Half of Expected Metric Value at Higher Thresholds

[darkterminal92]

We have successfully completed the PoC for autoscaling using KEDA + Prometheus + JVM heap metrics.

✅ The stack is fully automated

✅ JVM probe metrics are exposed correctly

✅ KEDA PromQL trigger is functioning

✅ Autoscaling can be reproduced on demand (e.g., when threshold = 30)

However:

When the threshold is set to 80 (representing 80% heap utilization), scaling does not trigger — even though Prometheus clearly shows values > 0.80.

The issue appears to be confined to how the metric is interpreted by the HPA via KEDA’s external metrics adapter.

sum(jvm_memory_used_bytes{area="heap", app="abc-service"})
/
sum(jvm_memory_max_bytes{area="heap", app="abc-service"})

Prometheus Result:

0.88 (≈88% heap utilization)

KEDA Trigger Configuration:

threshold: "0.80"

HPA Observed External Metric Value:
~0.40–0.45 (approximately half of expected value)

Problem Description

Despite Prometheus showing >80% heap utilization, the HPA receives a value approximately half as large.

This suggests a potential:

Unit conversion issue (e.g., milli-units)

Averaging across pods

Metric normalization inside KEDA external-metrics adapter

Implicit scaling factor before exposing metric to HPA

We have collected logs to demonstrate the discrepancy:

/tmp/hpa_watch*.log

These logs clearly show:

Prometheus value > 0.80

External metric value ≈ 0.40

No scaling triggered when threshold = 80

Scaling works immediately when threshold = 30

Reproduction

Deploy service with 2 pods

Generate heap pressure

Observe:

Prometheus → >0.80

HPA external metric → ~0.40

Lower threshold to 30

Scaling triggers immediately

🎯 Expected Behavior

If Prometheus returns 0.88, and threshold is 0.80, HPA should trigger scaling.

🧪 Environment

Kubernetes: (add version)

KEDA: (add version)

Prometheus: (add version)

JVM: (add version)

Pods: 2 replicas

[nicknikolakakis]

This is a known behavior. KEDA divides the metric value by the number of pods when exposing it to HPA as an external metric. This is by design because HPA's AverageValue target type calculates currentMetricValue / currentReplicas.

Why you see ~0.44 instead of 0.88

You have 2 pods. KEDA returns the raw Prometheus value (0.88), but HPA with AverageValue divides by replica count:

0.88 / 2 pods = 0.44

This is why it works at threshold 30 (0.30) since 0.44 > 0.30 triggers scaling, but not at 80 (0.80).

Fix

Option 1: Set the threshold to account for division

If you want scaling at 80% heap, set threshold to 0.80 / maxReplicas. But this is fragile as it changes with replica count.

Option 2 (recommended): Use metricType: Value instead of the default AverageValue

Note: metricType: Value is supported since KEDA v2.7+. Make sure your KEDA version supports it before applying.

triggers:
  - type: prometheus
    metricType: Value          # This prevents division by pod count
    metadata:
      serverAddress: http://prometheus:9090
      query: |
        sum(jvm_memory_used_bytes{area="heap", app="abc-service"})
        /
        sum(jvm_memory_max_bytes{area="heap", app="abc-service"})
      threshold: "0.80"

With metricType: Value, HPA compares the raw Prometheus result directly against the threshold without dividing by replica count. Note that metricType must be at the trigger level, not inside metadata.

Verify

After applying, check:

kubectl get hpa -o yaml | grep -A5 "external:"

The metric value should now match what Prometheus returns.