-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Description
Report
When using KEDA with the following ScaledObject configuration (Helm template below), fallback is triggered if metrics cannot be fetched. However, if minReplicaCount is set to 0, the deployment does not scale from 0 to 1 as expected when fallback is active.
fallback:
failureThreshold: 1
replicas: 1
behavior: currentReplicasIfHigher
minReplicaCount: 0
maxReplicaCount: 6
triggers:
- type: prometheus
metricType: AverageValue
metadata:
serverAddress: http://prometheus-operated.monitoring.svc:9090
metricName: redis_sidekiq_queue_backlog_total
query: sum(redis_key_size{key=~"{{ $scaler.queueList }}",namespace="....",service="..."})
threshold: "{{ $scaler.queueBacklog }}"
Expected Behavior
When fallback is triggered and replicas: 1 is set, deployment should scale from 0 to 1.
Actual Behavior
- Fallback is triggered (confirmed in events/logs).
- Replica count remains at 0, does not scale up to 1.
Steps to Reproduce the Problem
- Deploy ScaledObject with minReplicaCount: 0 and fallback as above.
- Cause metric source to fail (scale down prometheus).
- Observe fallback is triggered but no scaling from 0 to 1.
Logs from KEDA operator
ERROR 2025-11-07T15:05:01.443182024Z 2025-11-07T15:05:01Z DEBUG scale_handler Getting metrics and activity from scaler {"scaledObject.Namespace": "....", "scaledObject.Name": "....", "scaler": "prometheusScaler", "metricName": "s0-prometheus", "metrics": [], "activity": false, "scalerError": "Get \"http://prometheus-operated.monitoring.svc:9090/api/v1/query?query=sum%28tasks_held%7Bprotocol%3D%22s3%22%7D%29&time=2025-11-07T15:05:01Z\": dial tcp: lookup prometheus-operated.monitoring.svc on 10.56.0.10:53: no such host"}
ERROR 2025-11-07T15:05:01.443185323Z 2025-11-07T15:05:01Z ERROR scale_handler error getting scale decision {"scaledObject.Namespace": "...", "scaledObject.Name": "....", "scaler": "prometheusScaler", "error": "Get \"http://prometheus-operated.monitoring.svc:9090/api/v1/query?query=sum%28tasks_held%7Bprotocol%3D%22s3%22%7D%29&time=2025-11-07T15:05:01Z\": dial tcp: lookup prometheus-operated.monitoring.svc on 10.56.0.10:53: no such host"}
ERROR 2025-11-07T15:05:01.443188010Z github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).getScalerState
ERROR 2025-11-07T15:05:01.443214917Z /workspace/pkg/scaling/scale_handler.go:815
ERROR 2025-11-07T15:05:01.443217848Z github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).getScaledObjectState.func1
ERROR 2025-11-07T15:05:01.443220922Z /workspace/pkg/scaling/scale_handler.go:662
ERROR 2025-11-07T15:05:01.443236562Z 2025-11-07T15:05:01Z DEBUG scale_handler Removing entry from ScalersCache {"key": "scaledobject....."}
ERROR 2025-11-07T15:05:01.443242155Z 2025-11-07T15:05:01Z DEBUG scale_handler scaler error encountered, clearing scaler cache {"scaledObject.Namespace": "....", "scaledObject.Name": "...."}
ERROR 2025-11-07T15:05:01.443247830Z 2025-11-07T15:05:01Z DEBUG events Get "http://prometheus-operated.monitoring.svc:9090/api/v1/query?query=sum%28tasks_held%7Bprotocol%3D%22s3%22%7D%29&time=2025-11-07T15:05:01Z": dial tcp: lookup prometheus-operated.monitoring.svc on 10.45.0.20:53: no such host {"type": "Warning", "object": {"kind":"ScaledObject","namespace":"...","name":"....","uid":"....","apiVersion":"keda.sh/v1alpha1","resourceVersion":"1762464509434479012"}, "reason": "KEDAScalerFailed"}
ERROR 2025-11-07T15:05:01.443252113Z 2025-11-07T15:05:01Z DEBUG scaleexecutor ScaleTarget will fallback to Fallback.Replicas after Fallback.FailureThreshold {"scaledobject.Name": ".....", "scaledObject.Namespace": "....", "scaleTarget.Name": "...."}
ERROR 2025-11-07T15:05:01.543416968Z github.com/kedacore/keda/v2/pkg/scalers.(*prometheusScaler).GetMetricsAndActivity
KEDA Version
2.18.1
Kubernetes Version
1.33
Platform
Google Cloud
Scaler Details
No response
Anything else?
We are currently unable to set minReplicaCount to 0 in our KEDA ScaledObject configuration. If we do set it to 0 and Prometheus goes down, our deployment can potentially remain at 0 replicas until Prometheus is available again—even if fallback is triggered. This means our application may not recover automatically when metrics cannot be fetched.
Thank you for your help and consideration!
Metadata
Metadata
Assignees
Labels
Type
Projects
Status