Can't reuse pip-compile cache between platforms

[si14]

I'm not sure if it's a feature request or a bug, but we have a bit of a problem. README recommends running pip-compile on target platform. For us it means running it in docke and on developers' macbooks, as we need locking on both. However, if we do it naively it's very slow, as pip-compile has to redownload all packages every time. We tried to point pip-compile at an explicit cache folder and reuse it between host and docker environments, but apparently pip-compile caches a "trimmed" set of dependencies in depcache-PYTHON_VERSION.json (that is, filtered by platform), which means platform-specific dependencies (appnope from ipython in our case) are missing from host-resolved dependencies if we run pip-compile on host after running it in docker.

Environment Versions

1. OS Type: MacOS and linux
2. Python version: 3.8
3. pip version: 20.2.2
4. pip-tools version: 5.3.1

Steps to replicate

1. create a requirements.in with just ipython
2. create a docker container with pip-tools in it, mount an external cache directory, and run pip-compile inside it
3. run pip-compile on host with the same cache directory

Expected result

appnope should be in requirements.txt

Actual result

it isn't there if pip-tools was run in docker first

[glyph]

It seems like this is related to the discussion in #2051 .

I am attempting to manage a multi-platform project's dependencies with pip-compile, and it is… fighting me. Right now, sticking this into a shell script is my recipe:

pip-compile --strip-extras --quiet --upgrade --allow-unsafe --no-emit-index-url \
            --extra dev \
            -o dev-requirements.txt
pip-compile --strip-extras --quiet --upgrade --allow-unsafe --no-emit-index-url \
            --constraint ./dev-requirements.txt
pip-compile --strip-extras --quiet --upgrade --allow-unsafe --no-emit-index-url \
            --extra gui \
            -o "$(
            python3 -c '
import platform
print(platform.platform().split("-")[0])
'
)-gui.requirements.txt" \
            --constraint ./dev-requirements.txt\
            --constraint ./requirements.txt

Arguably what I'm doing here is emitting a series of constraints files, not really requirements files, and then I should pip install with -c rather than -r. But that only solves half the issue: right now I need to run pip-compile itself on all possible configured target platforms in order to generate separate files, and then come up with a naming convention to manage those.

Ideally, pip-compile could emit a single file while pretending that all platform specifiers evaluate to true, but continue doing what it's doing in terms of relaying those specifiers into the output. Then I could have one file with ewmh==0.1.6 ; sys_platform == "linux" and pyobjc-framework-cfnetwork==11.0 ; sys_platform == "darwin" without needing to come up with my own idiom for separating them into different files.