Description
Description
What problem are you trying to solve?
Karpenter currently has very limited support for Instance Savings Plans, which requires manual configuration of a NodePool with weights and limits to conform to the shape of the Savings Plan. Savings Plan are regional, which means that Karpenter needs to be aware of the Savings Plan in order to accurately make pricing decisions. Karpenter today is not smart enough to understand these savings, which is why the docs suggest manual NodePool weighting.
This is troublesome when multiple EKS clusters exist in a single region with a Savings Plan. There is no correct way to configure two or more Karpenter instances to be aware of a savings plan and its limits in this situation. The best we can do is to manually slice the savings plan into shards and configure each cluster with part of the savings plan, but that is toilsome, and leads to inefficient utilization of the savings plan.
Example: We buy a Savings Plan for m6i in us-east-1. We have two EKS clusters of equal sizes, "A" and "B". Configure Karpenter with a manually weighted NodePool with a limit that is half of the Savings Plan capacity for each cluster.
Scale up happens in Cluster A, but not in Cluster B. Cluster A now is using 100% of it's half of the Savings Plan. Cluster B is only using 10%.
More scale up happens in Cluster A, and now the next provisioned nodes are not m6i because the weighted NodePool is full. Now these nodes are out of the savings plan and full cost. However capacity still exists in Cluster B, which is paid for and wasted.
This scenario gets worse the more EKS clusters exist in a region.
What is needed is first-class support for Savings Plans. Karpenter needs to be aware of what Savings Plans exist in a region, and how utilized they are. This would enable Karpenter to coordinate with other EKS clusters and fully utilize Savings Plan. It's unclear if AWS has a good API to query Savings Plans, so this may require some sort of resource or configuration to exist that Karpenter can query. Definitely open to suggestions on how to implement this!
How important is this feature to you?
Very important, lacking this support is causing us to avoid ISP entirely.
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment