Is --no-cache-dir really needed?

[jdemeyer]

I saw in the README.md that you need --no-cache-dir in some cases. Is this really true? If true, that is quite annoying since pip install should "just work".

[embray]

I think this only applies to users who would be installing cypari from PyPI. There are two reasons for this:

1. If you already have the same version of a package installed pip won't reinstall it for you (hence the `--force-reinstall).

2. For packages with compiled modules, part of the point of building the wheel (part of the point of the wheel format in general) is that it can contain binaries. It's possible, for example, to upload pre-compiled binary wheels for different platforms to PyPI. When you install from a source tarball it will build the package (if possible) and cache the result as a binary wheel for your system to prevent having to rebuild the same package every time you install it (in, say, a virtualenv).

That said, I guess if you've changed your pari installation (and assuming aspects of cypari's binaries depend on the pari version, which makes sense), there's no way for Python or pip to know that. So if you reinstall cypari using pip install cypari2 it has no other way to know other than --no-cache-dir that the cached build will not work anymore against your new pari installation.

This only applies to users who would be installing from PyPI. In development it's irrelevant since you'd be running pip install . (or better yet pip install -e . if that works), and would have to be rebuilding the sources when changes are made anyways.

[jdemeyer]

Thanks for the info.

I guess there would also be problems if the user has two copies of PARI in some virtual environment (like Sage, conda, ...) and wants to pip install cypari2 in both copies, linking to a different PARI library. If I understand things correctly, the first time this would cache a wheel (using the PARI from that virtual env) which would be used by the second copy.

[embray]

Yes, that sounds right.

Another possibility for the future, but unfortunately not available to us yet is a custom "platform tag"in the filename. This was originally specified here, though I'm releaizing PEP-425 is a bit out of date and should be updated here. The "platform tag" is not just limited to what's returned by distutils.util.get_platform() as that PEP states. Really this can be anything (see for example PEP-513 describing the manylinux1 platform tag.

I know there's been some talk about describing a protocol for custom platform tags (so in theory we could have platform tags that relate to different pari installations), but so far there's no consensus on that, and it's obviously tricky to get right.

[jdemeyer]

Do you happen to know whether there is an easy way for the package to prevent pip from using the cache?

[embray]

I don't think there is, no. Nor do I think it should by default.

You've gotten me thinking though--it would be nice if packages could specify in their metadata some files that impact the build of binary modules. To an extent this could be generated automatically too--for example any libraries that extension modules are linked to could be automatically included in such a list. When pip builds a wheel that is saved in the wheel cache, it could also record the hashes of those dependent files on the user's system. Then force a rebuild upon pip install if any of those files have changed.