when and where is the best way to download the model weights?

[ynie]

We want the model be part of the docker image, but we are struggle to find a place to run the python command to download them. Where can we do it in the cog.yaml file? Thanks!

[humphd]

This is exactly my question. I've read all through the docs and examples, and I'm really surprised this isn't dealt with more clearly.

In the docs it warns against doing this in setup():

It's best not to download model weights or any other files in this function. You should bake these into the image when you build it.

How? I know how I'd do this if I was doing the Dockerfile myself, but the point of cog seems to be that I offload this process to the tool itself.

Then in the Resnet example, there seems to be some magic happening with the file resnet50_weights_tf_dim_ordering_tf_kernels.h5 being downloaded before the build and then setup() having local access to it.

How and where should one do this? Can the docs be updated to give more explicit guidance?

[ynie]

I really like the idea of Replicate, but I rarely be able to find any support of this company. It’s great for prototyping, but I would not recommend for production.

[humphd]

Attempting to answer my own question, it appears that cog will attempt to locate large (>10M) files that don't have certain ignored extensions (see FindWeights()) and then COPY these into /src in the built image.

So I guess I need to download my model and put it beside (or below) my code and it should "Just Work."

This is cool, but opaque and not well documented. It would be great to showcase this as a feature in the docs.

[ynie]

Oh wow, it seems like these weights will be re-uploaded each time. Is it the same for you?

[humphd]

I haven't tried this yet, still reading. But I would assume Docker will cache this layer and re-use on subsequent pushes.

[zeke]

Hey @ynie and @humphd. Sorry for the delay getting an answer to you here. This is definitely something that should be better documented.

A common approach for downloading weights is to create a standalone script (typically script/download_weights in the style of GitHub's Scripts to Rule Them All), then invoke that script as part of your build and deployment process.

cog run script/download_weights
cog predict ...
cog push ...

Cog will include any files that are present in the project directory (and subdirectories) in the compiled Docker image, unless they are explicity ignored patterns in your .dockerignore file. So you can download the files wherever you want and name them whatever you want.

I know the @replicate/models team has also been using pget more and more for downloading weights. as its apparently faster and/or more reliable than alternatives like cURL and wget. Here's an example of how to install pget it in a cog.yaml file:

build:
  run:
    - curl -o /usr/local/bin/pget -L "https://github.com/replicate/pget/releases/download/v0.3.1/pget" && chmod +x /usr/local/bin/pget
   
predict: "predict.py:Predictor"

[humphd]

Thanks for confirming how this works, @zeke. Appreciated.