Use pre-baked AMIs with installed packages and scripts

[villasv]

This would improve two aspects: initialization times and code reuse (worker, scheduler and interface can be identical images).

[villasv]

It is so tempting to dive into this. On the other hand, auto-scaling of Celery workers is not as shiny as in other domains like microservices. In fact, nowadays this seems the least relevant feature of this stack. Initialization times are always best when minimal, but are not really important.

The best motivator would be code reuse. It really is cumbersome to maintain three ~almost~ identical launch configurations. I believe that reusing the launch configuration is not enough of a motivator to a radical approach such as containers.

[villasv]

Code reuse can be achieved with this trick, so I don't intend to go on with images.

[villasv]

Reopening since Fargate makes adopting containers even more attractive (no EC2 left to manage). But I wonder if it's worth the wait for EKS to be GA and battle tested.

[villasv]

A year has passed, a decision must be taken. My opinion today is basically the same: it would be awesome.

But there are other areas I need to improve first. After many months of stalemate, I've decided to stick with EC2 until every other important aspect is polished and only then consider a RFC with pre-baked AMIs and maybe another RFC with containers.

[villasv]

After discussion on #166 I've decided to revive this issue. The WaitCondition shared metadata trick is no longer viable, the scripts were immense and were moved out of the tamplate to a shell script.

Yet another year has passed (since this was closed), and the areas I needed to improve are now better polished.