cloud
cloud copied to clipboard
Re-use images (only changing parameters)?
My current workflow is to use the gcloud CLI to submit jobs with a common base image and differ the parameters (used in a CI setting). I like that in tensorflow cloud that the machine types and accelerator types are expressible as code, but I don't want to rebuild the image (and re-upload) the image each time I train.
Is there anyway to re-use the same image (but just supply a different 'trainer' file)? What I'd like to do is to push the trainer file + hyper parameters (as YAML or JSON) and keep the image the same across runs (except when my dependencies change).
I see that you can specify a custom base image, but AFAICT a new image would still be built / pushed.
Thank you for the issue @tc-wolf . This is a great question and I see that this is a very useful and important feature to have. Let me look into adding support for this.
Thank you for looking into this :)