Caching built C extensions

[DanielShaulov]

We are trying to get a single executable that has all the extension modules that we need as "build-in", and we found that running pip_install with flags to make it not use wheels for the packages that have C extensions works well:

exe.add_python_resources(exe.pip_install(["-r", "requirements.txt", "--no-binary", "psutil,grpcio"]))

The thing is - it takes a very long time to compile (specifically - grpcio is taking ~5m). We usually only change our own code (added with a read_package_root line after the line above), so it would be great if we could somehow not do the compilation again unless the requirements file changes (or we do some force cache cleaning).

To speed this up I can think of two options:

1. Caching the results of compilation on pip level - currently all the pip install have a tempdir as the target, but even changing it to a "constant" directory doesn't really fix anything because pip install still does the compilation again, even when the target directory already has it compile, I think because of https://github.com/pypa/pip/issues/9769, but the resolution for that seems to be stalled.
2. Building "in steps" - if we could have 2 different invocations of pyoxidizer build - one that would do everything up to and including installing the requirements, and then a separate invocation to just add our own sources and then compile the final app, that would solve everything for us, because we use a Makefile to run the build anyway, and can just rerun the first command only if requirements.txt changed, and run only the second phase if not.

Is there anything I can do in the pyoxidizer.bzl file to store PythonExecutable object in one target and then load it in a different target? Any other ways to improve this?

[hariom-qure]

It would be great if the caching is used for filesystem:relative mode atleast, im a total newbie at this but was hoping that it might be easier to do this...