Improve caching by reusing the temporary "cibw-run-" folder

[yeicor]

Description

Hi! Thanks for your hard work maintaining this great utility.

I am building a Python wrapper for a large C++ project and I realized that I am recompiling the sources many times when the object files should be cached.

After some investigation, I realized that the locations of the Python headers change each time I run cibuildwheel, causing tools like Ninja and ccache to invalidate their caches and recompile the complete bindings. The provided links show an example where the caches should have kicked in but they do not. Note that I am only using Pyodide, and I don't know if other platforms are affected.

I tested a few quick changes that make the caches work:

• tmp_path = Path(os.getenv("CIBW_RUN_TMP_PATH") or mkdtemp(prefix="cibw-run-")).resolve(strict=True) and disable finally block in __main__.py.
• Add exists_ok=True to a few .mkdir() calls in platforms/pyodide.py.
• Disable exists assert in util/python_build_standalone.py (run the whole function as usual).

After this, setting CIBW_RUN_TMP_PATH to something like $HOME/.cache/cibw-run would reuse caches between different runs (the directory must already exist).

This would need more work for other platforms, but I would appreciate it if you consider implementing this as a feature of this project (and maybe even default to reusing $HOME/.cache/cibw-run as the temporary path, using subfolders for each platform as required).

Build log

https://github.com/yeicor/OCP-wasm/actions/runs/15517516204/job/43686393533

CI config

https://github.com/yeicor/OCP-wasm/actions/runs/15517516204/workflow