automlbenchmark
automlbenchmark copied to clipboard
Workflow to build and publish docker images
For the stable
branch only, it would be useful to have a workflow that takes a framework definitions file, and from there:
- list all frameworks for which to build a docker image.
- builds images.
- test each of them with
validation
dataset (1 fold each, 10 min duration). - publish the images to docker hub.
What to do
- when it fails building a docker image.
- when an image fails on the
validation
dataset.
We probably want to deploy only on a push
but verify the builds work on a pull request
. Then we can sort out any errors before the push to stable and once its pushed to stable it should build and deploy. The jobs might require a bit more resources than an average unit test, so we will have to see which providers allow us to do these more intensive (but infrequent) jobs.