Restructure Cog configuration options for base image selection

[mattt]

trafficstars

Related to https://github.com/replicate/cog/pull/1205#issuecomment-1634961977

As proposed, the changes in #1205 would have the following top-level config fields (among others):

1. gpu
2. cuda
3. cudnn
4. use_cuda_base_image

Although they're at the same top-level of cog.yaml, they interact hierarchically:

• cuda and cudnn are used to specify the CUDA base image
• ...but those are ignored if use_cuda_base_image is false
• ...but that's moot if gpu is false

(And if we supported a custom base image, that could invalidate all of those settings)

A better solution would express these dependencies in the config structure itself. As we saw in #1010, it's possible to support advanced features in progressive way that doesn't compromise simple syntax.

Here's a rough sketch of what that might look like:

• At the top level, the user has the option to specify whether to use a specific base image or to have Cog decide
  • If we let Cog decide, the next option down is whether to use CUDA base image or not
    • If we choose CUDA base image, then we allow user to override CUDA and cuDNN versions.

Not at all committed to this spelling, but maybe something like:

base: nil # default, implied by omission, Cog automatically chooses base image (python/*-slim)

base: <custom/image> # user-provided override

base:
  cuda: 11.8 # use CUDA base image
  cudnn: 8.6

We could introduce this as a minor revision to the current cog.yaml schema, deprecating but still supporting the aforementioned GPU-related fields.

Thoughts, @andreasjansson, @bfirsh, @technillogue?

[bfirsh]

Oooh, interesting. Custom images could be a can of worms, as Andreas hints. I'll let this percolate. :)

To paint a bike shed, if we do want to go in this direction: "base" is a new term, I think. In Docker-land, it's typically "from" (in Dockerfile) or "image" (in docker-compose.yaml, and other places). One could argue we want to intentionally distance ourselves from Docker, though.

[prassanna-ravishankar]

@bfirsh what about parent ?