Description

What problem are you trying to solve?

I've been talking with some folks about the use of minValues when it comes to selecting instance types and increasing the flexibility when Karpenter launches with spot. The core problem here is, without more context, Karpenter can overpack your pods onto the largest instance type size, reducing the flexibility of your spot instance launch, and causing you to continually be launched in a really bad instance type pool.

minValues is a great solve for this because it forces the scheduler to enforce a minimum "flexibility" on a requirement; however, the current configuration enforces this as a hard requirement. This means that anyone who's looking to implement this flexibility is forced to create a fallback NodePool to ensure that they don't get put into a situation where they are unable to schedule pods when capacity is constrained for the instance types in their NodePool.

One proposed solution to this problem is to create a whenMinValuesUnsatisfiable that has the values DoNotSchedule, ScheduleAnyways, and UseUnavailable. The values would have the following defintiions

• DoNotSchedule: This is a hard requirement, it requires that the minValues in the requirement is satisfied by available offerings
• ScheduleAnyways: This is a soft requirement, it attempts to satisfy the minValues requirements with available offerings. If it can’t satisfy it, then it will attempt to support as many values as possible for the requirement, but it will still allow the NodeClaim to launch without satisfying the requirement
• UseUnavailable: This is a soft requirement, it attempts to satisfy the minValues requirements with available offerings. If it can’t satisfy it, it attempts to satisfy the minValues with unavailable offerings. If it can’t satisfy the minValues with unavailable offerings, the NodeClaim will fail to schedule and we will attempt other NodePools

How important is this feature to you?

This would reduce the configuration burden for folks who are using the minValues feature to increase the flexibility of their NodePools.

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment