Check for cheaper instances and allocate at less periodic interval

[gamefundas]

I just configured the project and this sounds fantastic compared to spot fleets which are harder to configure or at least isnt configuration less like AutoSpotting.

Now I did go through the FAQ about this and do see the mention of this not being supported but I guess you have all the building blocks to make it work if needed. Why not at a configurable frequency on the AS group level allow for cheaper spot instances replacing the old one when there is a better price. This can be less periodic may be people can decide to choose less frequency, perhaps once a week or something but this is going to be super useful for non-prod environments where one off outages for this kind of auto scaling down to cheaper instances over the weekend will add so much value.

Unless I am missing any complexities involved here this can prove to be very useful and distinguish this project from all other offerings.

[cristim]

Thanks for your request.

This is no longer a problem, since AWS introduced the fixed pricing the prices are pretty much flat.

Nevertheless, we do have the aggressive bidding policy which can be used to detect price increases, by default over 10% since the instance was launched.

[gamefundas]

@cristim but I am referring to a situation where a cheaper spot instance is now available since the last time Autospotting allotted one. It doesnt look like Lamda will every check an existing spot instance to see if a better rate is available than the one that was found earlier.

[cristim]

Indeed, it doesn't actively seek to replace the current instances with the cheapest available, but using the aggressive bidding policy you can get roughly the same results.

The spot instances that got 10% more expensive will be terminated by the market when the bid price is exceeded, the group will launch a replacement on demand instance that is going to be replaced by AutoSpotting with the current cheapest spot instance.

I actually had plans to implement it that way but it became pretty much irrelevant when AWS made the spot pricing stable.

[gamefundas]

@cristim thanks for the explanation. Have couple of other questions around AutoSpotting behavior, hope I didn't miss reading in the docs or faq.

1. In case if I already have spot instances/fleet running through cloud formation then will Auto Spotting do anything to those or will simply ignore them because they are already spot instances.

2. What if I tag these spot instances as spot-enabled=true, will Auto Spotting manage them better in case of a aggressive policy?

Where I am going with this is, basically our VPC had only on demand/reserved instances so far. After running Auto Spotting most of them (tagged with spot-enabled flag true) got replaced by spot instances however soon the team is going to start redeploying with new cloud formations that will have spot instances to begin with. I am wondering if these new cloud formation spawned instances (if they happen to have spot enabled flag true) will conflict with Auto Spotting.

[cristim]

AutoSpotting only works with instances in Autoscaling groups, not for individual instances or spot fleets. The spot-enabled tag is supposed to be set on the group and doesn't have to be propagated to the instances.
The aggressive policy is only a matter of automatically setting the spot bid price to current price +10% for the spot instances launched by AutoSpotting itself. The bid price for instances in spot fleets is not going to be set the same way unless you implement it like that yourself.

AutoSpotting would currently not touch any running spot instances in the groups tagged with spot-enabled this may change in the future so I would recommend you to remove the spot-enabled tag on the groups configured to launch spot instances out of the box so that AutoSpotting wouldn't work against them at all, in case AutoSpotting changes this behavior later.
.

[gamefundas]

Perfect, we can follow that as a practice to not have spot-enabled flag for self managed spot instances.

As a cost saving measure we are also looking to size instances (vertical scaling) according to their utilization. Is there any way AutoSpotting will be able to help here. For instance if I manage to add a tag to ASG to indicate a different (scaled up/down instance type) via the "allowed_instance_types" flag, will AutoSpotting try to find an appropriate spot instance, more close to the newly defined instance type in terms of price and ec2 capacity?

[cristim]

No, there's no functionality for that at the moment but it would be nice to have, thanks for the idea.

We're actually also in the process of doing this type of vertical scaling exercise and we have a tool able to come up with an instance type recommendation based on a Machine Learning model, we may integrate this into AutoSpotting at some point.

[gamefundas]

@cristim may be a little intrusive measure for now but is there any way AutoSpotting can be retriggered through some other action. I was thinking perhaps if I find instances in an autoscaling group under/over utilized for x no. of days on stretch then I can set the desired instance in ASG to zero, which will get rid of all the instances, then set "autospotting_allowed_instance_types" to a lower or better type for that ASG and then reset ASG desired instance count to original value. Wondering if this would get me to what I need.

Long run it would be great to see AutoSpotting support this feature.

[cristim]

I've also been trying to articulate this. I doubt it's doable with the current implementation but it should not be so hard to do, assuming that we have a good ML model that can suggest good instance types for the current group.

The way I would likely do it is to have another ML-powered tool generate and later maintain the allowed instance types override tag set on each group, but only using exact names, without wildcards. Maybe give a couple of candidates to iterate over.

AutoSpotting would also have a new configuration flag that instructs it to just iterate over those instead of using the current compatibility check logic. This should be relatively easy to implement and would ensure that any newly launched spot instances are rightsized.

In addition to this, we can have another parameter that tells AutoSpotting to actively terminate spot instances of types not matching the allowed list. This is also doable with relatively small effort.

The main question is which bid price to use? If we use the on demand price of the initial instance type we may fail to launch larger types.

What do you think?

[gamefundas]

Like the overall approach. With respect to picking the new instance type, some kind of prioritization can be employed to get the nearest match (lower or higher depending on scaling up or down) and then use that instance type for bid price as now we are bound to operate the cluster with that instance.

The old instance was deemed to be unfit and hence the reason for switch. In this case its possible that the cost increases compared to the previous instance when scaling up but then its a needed changes. On the other hand scaling down to lower instance but the spot price increases then its not reasonable enough to make the change.

[lenucksi]

I guess I've read some discussions on prioritization of instance types. Some revolved mitigating around risk involved in launching newer instance types leading to problems with images optimized to older instance types.

Currently the launch list is just filtered for compatibility and sorted by price. One could introduce (an) additional sort factor(s) / weighting of compatibility factors (or other factors) and price to approach this.
The approach to use an external whitelist-provider for instances also sounds interesting to me, as it would keep autospotting small and allow more or less random models to be used.

[gabegorelick]

I've run into the lack of this feature. While the new spot pricing model has certainly reduced price volatility, it still exists. For example t3a on-demand instances are always cheaper than their t3 counterparts. But the spot price of t3a instances fluctuates a lot: sometimes they're cheaper than t3's and sometimes they're more expensive. By picking one or the other, AutoSpotting can lock you into the wrong choice until the instance terminates.

I don't know enough about finance to say what a better bidding strategy would be, but it seems like incorporating price volatility into the equation might make sense for users that want a more consistent price. Alternatively, constantly replacing instances may be a better strategy for users seeking better deals. However, that strategy may be counterproductive since swapping instances incurs an extra cost for the period of time when both instances are online.

With all this being said, I can't fault AutoSpotting for keeping it simple. But there are certainly cool things it could do.

[cristim]

Thanks, I think one way out of this situation would be to force a spot replacement as soon as the spot price of another candidate instance type becomes marginally better on the long run.

I'm not sure how the comparison logic would look like but I am open to suggestions.

Let's reopen this and come up with a way to make it work.

[cristim]

@gamefundas @gabegorelick: is this issue still of interest to you? I may implement it if people still need it.