pypa
Add option to specify where the sdist tarfile should be expanded
Currently, if no -s or -w option is given, a sdist is created which is then extracted in a /tmp/build-sdist-xxx directory. It would be useful to have an option to specify this directory. That's important when compiling large C extensions which can benefit from using ccache. When a random temporary directory is created then ccache is of not use.
In other words, my request is to have an option to specify the directory sdist_out in this line: https://github.com/pypa/build/blob/f5d5f15f1e8a4e501ad5044cd2d82e0c7995171c/src/build/main.py#L238 . Obviously the default should be the current behaviour and only when the option is specified that directory should be used.
I know that the current -w option helps in that sense, but it is desirable to have off-source builds since the extension compilation can potentially create many files and mess around with the sources.
I assume the goal here is to build multiple wheels from the same sdist - what's the user flow gonna end up looking like with the proposed option? Will you build --unpack-sdist-at foo once then cd foo and build -w x times? I don't think that's much of an upgrade from build -s -> tar xzf -> build -w * x in terms of usability and a downgrade in terms of clarity.
The use case if for developers which make frequent changes to the C/C++ code. Our code base can take a significant time to compile. The idea is that build --unpack-sdist-at foo -o dist src would create both a sdist and a wheel in dist that can be tested, then more changes are done in the C/C++ part of src and the same build command is run again. Both the sdist and the wheel will be different this time, but the wheel compilation will hopefully be faster because ccache recognizes the same hash of the C/C++ files. Currently that's not the case because the path of the files is part of that hash.
Ah right, that makes more sense - you don't actually want to keep foo around, you just want the path to be deterministic between builds in order to hit the cache. Hmm, I don't know if adding a new option is the best option. Perhaps build can default to using $TMPDIR/pyproject-build/$(basename $PWD) if it's unoccupied and fall back on mkdtemp and emit a warning if it isn't?
Yes, the point is having the path to be deterministic.
I actually find the default of using mkdtemp a good one :) In the sense that a change of the default behavior of build can surprise users. In fact, having a random working directory is probably a good idea to catch certain bugs in the build system due to hardcoded paths and the like. So from a general perspective I find that a good choice.
The thing is that for certain cases like ours, in which the developers know what they are doing, it would be useful to have some control of the working directory.
Hello, I would like to work on a feature for specifying the sdist extraction directory and the isolated environment directory to speed up ccache usage. Would a PR implementing this be favorably looked upon by the maintainers?
https://github.com/pypa/build/blob/a25e35b5470fdb3c722c06dc04113d850d93e748/src/build/main.py#L272
https://github.com/pypa/build/blob/a25e35b5470fdb3c722c06dc04113d850d93e748/src/build/env.py#L76
