keepsake icon indicating copy to clipboard operation
keepsake copied to clipboard

Use venv for Python development

Open hemildesai opened this issue 4 years ago • 12 comments

Addresses https://github.com/replicate/replicate/issues/424

hemildesai avatar Dec 22 '20 21:12 hemildesai

Not sure why CI / Test (Python) (3.6, ubuntu-20.04) is failing. The error seems to be No matching distribution found for tensorflow-cpu==2.4.0 (from -r requirements-test.txt (line 4)).

hemildesai avatar Dec 24 '20 10:12 hemildesai

Yeah, super weird isn't it. I restarted the test a couple of times so it's a real failure.

Maybe it's using a different version of pip because it's inside a virtualenv? I'm trying to figure out what might be different in this installation.

bfirsh avatar Dec 24 '20 18:12 bfirsh

Thanks so much for the contribution btw! I'll do some poking around later today if I get time...

bfirsh avatar Dec 24 '20 18:12 bfirsh

Cool, thanks for looking into it. Let me know if I can help.

hemildesai avatar Dec 24 '20 18:12 hemildesai

@bfirsh it looks like the pip version is the culprit - https://github.com/tensorflow/tensorflow/issues/39130#issuecomment-623335820

hemildesai avatar Dec 24 '20 18:12 hemildesai

Hah, oh god. Nice find. 🙄

bfirsh avatar Dec 24 '20 19:12 bfirsh

Several times during this project we've thought "ok we need to burn all Python packaging to the ground and create a new packaging system" but then reminded ourselves we should keep our eyes on the ball, haha. This is also why most of the codebase is in Go -- so we don't have to deal with Python's godawful packaging/dependencies. cc @dscape

bfirsh avatar Dec 24 '20 19:12 bfirsh

Yeah, sometimes with issues like this, it can be quite frustrating.

hemildesai avatar Dec 24 '20 19:12 hemildesai

Hey @bfirsh, let me know if this is good to go or if there are any changes required.

hemildesai avatar Jan 07 '21 18:01 hemildesai

Thanks so much @hemildesai! We're just in the process of shipping a release right now so I don't want to change up the development environment while we do that. But, as soon as we've shipped it, this is first thing I'm taking a look at. :)

bfirsh avatar Jan 07 '21 19:01 bfirsh

Sorry for being slow here -- I had a quick look at this the other day. I think it's a bit confusing that the Python library is installed locally in a virtualenv, but the binary is still installed globally. That'll make it easy to get into a situation where different versions installed in two different places. I wonder if we can install the binary inside the venv in development?

bfirsh avatar Feb 15 '21 20:02 bfirsh

@bfirsh Sure. Can you point me to where the binary is installed so that I can change it to be installed in the venv?

hemildesai avatar Feb 19 '21 16:02 hemildesai