cog icon indicating copy to clipboard operation
cog copied to clipboard

Vendoring (take 2)

Open andreasjansson opened this issue 1 year ago • 2 comments

This PR vendors all Python packages using the vendoring library.

Vendoring allows users to install packages that rely on versions of libraries that are in conflict with Cog's dependencies (e.g. Pydantic, see https://github.com/replicate/cog/issues/1562, https://github.com/replicate/cog/issues/1384, https://github.com/replicate/cog/issues/1186, https://github.com/replicate/cog/issues/1586, https://github.com/replicate/cog/issues/1336, https://github.com/replicate/cog/issues/785).

Vendored packages are gitignored and are sync'd by the default make rule.

Closes https://github.com/replicate/cog/issues/409

andreasjansson avatar Apr 15 '24 15:04 andreasjansson

Had a quick look at this and I think it could work. A couple of thoughts.

  1. I don't think we actually need to check in the files, do we? We have everything pinned in vendor.txt, so maybe let's have vendoring sync be part of the build process for the wheel. This will also help us ensure that we've nailed down any platform-specific stuff like cpython modules. As it is you've got a bunch of Darwin .sos checked in.
  2. You probably want to exclude everything in bin/ too. The shebang lines will be wrong and we don't ever need to use any of the deps as scripts.

nickstenning avatar Apr 16 '24 14:04 nickstenning

Great points @nickstenning. I've implemented your suggestions, much better!

andreasjansson avatar Apr 16 '24 16:04 andreasjansson

Closing in favor of #1687. I think we're close, but if we can't manage to get that working, this is probably the next best alternative.

mattt avatar Jun 24 '24 13:06 mattt