AWS EKS Prometheus and value_error

[mxw-sec]

Describe the bug
Running
krr simple --history-duration X I get the same output regardless.

WARNING Not enough history available for cluster arn:aws:eks:us-east-1:{ACCOUNT}:cluster/staging-karpenter. runner.py:235
WARNING If the cluster is freshly installed, it might take some time for the enough data to be available. runner.py:238
WARNING Enough data is estimated to be available after 2024-10-04 02:22:55, but will try to calculate recommendations anyway. runner.py:241
INFO Listing scannable objects in arn:aws:eks:us-east-1:{ACCOUNT}:cluster/staging-karpenter init.py:58
ERROR An unexpected error occurred runner.py:349
Traceback (most recent call last):
File "robusta_krr/core/runner.py", line 342, in run
File "robusta_krr/core/runner.py", line 288, in _collect_result
File "robusta_krr/core/integrations/kubernetes/init.py", line 531, in list_scannable_objects
File "robusta_krr/core/integrations/kubernetes/init.py", line 534, in
File "robusta_krr/core/integrations/kubernetes/init.py", line 63, in list_scannable_objects
File "robusta_krr/core/integrations/kubernetes/init.py", line 249, in _list_scannable_objects
File "robusta_krr/core/integrations/kubernetes/init.py", line 249, in
File "robusta_krr/core/integrations/kubernetes/init.py", line 172, in __build_scannable_object
File "robusta_krr/core/models/allocations.py", line 89, in from_container
File "pydantic/main.py", line 341, in pydantic.main.BaseModel.init
pydantic.error_wrappers.ValidationError: 1 validation error for ResourceAllocations
limits
invalid literal for int() with base 10: '3200e6' (type=value_error)

To Reproduce
run any command such as

krr simple --history-duration X

Expected behavior
Output Provided

Screenshots
If applicable, add screenshots to help explain your problem.

Are you interested in contributing a fix for this?
Yes/no. If yes, we will provide guidance what parts of the code to modify and help you.

Desktop (please complete the following information):

• OS: Ubuntu
• Browser [e.g. chrome, safari] N/A
• Version [e.g. 22] Latest

Additional context
Add any other context about the problem here.

[aantn]

Weird, I'm not sure what the problem is. The limits field is not an integer, it's a float field which makes the error especially weird.

Do you see any more information when running with --verbose?

[mxw-sec]

Nothing new when running with --verbose

``Running Robusta's KRR (Kubernetes Resource Recommender) v.1.15.0
Using strategy: Simple
Using formatter: table
[10:48:29] DEBUG An error occurred while checking for a new version runner.py:85
Traceback (most recent call last):
File "robusta_krr/core/runner.py", line 79, in
__check_newer_version_available
File "robusta_krr/core/runner.py", line 75, in __parse_version_string
ValueError: invalid literal for int() with base 10: ''

[10:48:30] DEBUG Creating kubernetes python cli monkey patches patch.py:10
DEBUG Found 4 clusters: init.py:493
arn:aws:eks:us-east-1:{Account}:cluster/karpenter-test,
arn:aws:eks:us-east-1:{Account}:cluster/test-cluster1,
arn:aws:eks:us-west-2:{Account}:cluster/indtest-eks,
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG Current cluster: init.py:494
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG Configured clusters: [] init.py:496
INFO Using clusters: runner.py:280
['arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter']
INFO Prometheus URL is specified, will not auto-detect a metrics service loader.py:55
INFO Trying to connect to Prometheus for prometheus_metrics_service.py:68
arn:aws:eks:us-east-1:{Account}:cluster/staging-karp
enter cluster
INFO Using Prometheus at http://127.0.0.1:9090 for cluster prometheus_metrics_service.py:97
arn:aws:eks:us-east-1:{Account}:cluster/staging-karp
enter
INFO Prometheus found loader.py:74
INFO Prometheus connected successfully for loader.py:47
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter cluster
DEBUG History range for runner.py:231
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter:
(datetime.datetime(2024, 10, 11, 5, 48, 31), datetime.datetime(2024, 10,
11, 10, 48, 31))
[10:48:31] INFO Listing scannable objects in init.py:58
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG Namespaces: * init.py:59
DEBUG Resources: * init.py:60
DEBUG Listing HPA-v2s in init.py:189
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG Found 3 HPA-v2 in init.py:221
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG Listing Deployments in init.py:189
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG Listing Rollouts in init.py:189
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG Listing DeploymentConfigs in init.py:189
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG Listing StatefulSets in init.py:189
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG Listing DaemonSets in init.py:189
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG Listing Jobs in init.py:189
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG Listing CronJobs in init.py:189
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG DeploymentConfig API not available in init.py:253
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG Rollout API not available in init.py:253
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG Found 2 Job in init.py:221
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG Found 1 CronJob in init.py:221
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG Found 3 StatefulSet in init.py:221
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG Found 6 DaemonSet in init.py:221
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
DEBUG Found 184 Deployment in init.py:221
arn:aws:eks:us-east-1:{Account}:cluster/staging-karpenter
ERROR An unexpected error occurred runner.py:349
Traceback (most recent call last):
File "robusta_krr/core/runner.py", line 342, in run
File "robusta_krr/core/runner.py", line 288, in _collect_result
File "robusta_krr/core/integrations/kubernetes/init.py", line 531,
in list_scannable_objects
File "robusta_krr/core/integrations/kubernetes/init.py", line 534,
in
File "robusta_krr/core/integrations/kubernetes/init.py", line 63, in
list_scannable_objects
File "robusta_krr/core/integrations/kubernetes/init.py", line 249,
in _list_scannable_objects
File "robusta_krr/core/integrations/kubernetes/init.py", line 249,
in
File "robusta_krr/core/integrations/kubernetes/init.py", line 172,
in __build_scannable_object
File "robusta_krr/core/models/allocations.py", line 89, in
from_container
File "pydantic/main.py", line 341, in pydantic.main.BaseModel.init
pydantic.error_wrappers.ValidationError: 1 validation error for
ResourceAllocations
limits
invalid literal for int() with base 10: '3200e6' (type=value_error
`

[mxw-sec]

I am running Krr on ubuntu 22.04.5 in WSL, I installed it via brew, and just updated to 1.16.0.. Still getting the same issue.

[arikalon1]

I think the 3200e6 is a legal limit in k8s, and krr doesn't know to parse it correctly
3200e6=3200000000, right?

The issue seems to be while handing a deployment

Can you try running this, and see if indeed you have that memory limit?
kubectl get deployments -A -o=jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.template.spec.containers[*].resources.limits.memory}{"\n"}{end}'

[arikalon1]

[moshemorad]

Issue was fixed in #361