control
control copied to clipboard
supergiant
2.1: Support for EC2 Spot Instances
It would be great if Supergiant would support provisioning EC2 spot instances.
As a first step the node_sizes entry of the kube config could be extended to accept entries in the form of "m4.large=0.05" which would indicate a spot instance with a maximum bid price of 0.05.
A far more complex implementation could do smart management of EC2 spot and on demand instances:
The Supergiant server would first try to provision spot instances, but if this fails, eg. because the maximum spot bid is too low or because spot capacities are exhausted, it would fall back to provisioning on demand instances of the same size.
It should then also continually monitor the instances to see if existing spot instances are forced to shut down (it could use EC2 spot instance shutdown notifications for that) and if associated spot requests are pending for a longer time, replace those with on demand instances as well.
It would also try to replace the on demand instances with spots, if the prices fall again or capacity becomes available.
I currently know of a standalone solution that try to achieve similar goals:
AutoSpotting: AutoSpotting works in the opposite way: It schedules a lambda function that looks through tagged autoscaling groups in your aws account and tries to replace them with spot instances. If those spot instances shut down, they automatically get replaces with on demand instances by the autoscaling group to restore the desired capacity.
While this might work with Supergiant a fully integrated solution would be more efficient and could give useful info in the dashboard, like price history and number of active spot vs on demand instances.
+1, my current kubernetes strategy and cost strategy hinges on our ability to use spot instances.
Wish list:
- Multi-AZ compat. to ensure best price (prices often vary across AZs) The idea would be if the price of an instance sky rocks in one AZ, we may be able to use another AZ to get the previous or even lower price.
- Termination handling - kubectl drain the nodes within the 120 second window.
- Cost analysis - Use historical pricing data to determine best "safe"-ish price to start at, with a user defined max.
- Ability to configure a "give up" period, where if we're unable to fulfill the spot fleet/requests, then on-demand instances are launched. If the request is eventually fulfilled after launching on-demand, than swap over to the spot instance.
@jordanrinke This prob will not be in the provisioning process. We will need to include this as a package on the capacity service, that has a API/UI element allowing for the adjustment of the spot prices.
