build icon indicating copy to clipboard operation
build copied to clipboard

Controlling location of isolated build environments?

Open gwerbin opened this issue 2 years ago • 8 comments

Is there a way to control exactly where isolated build environments are created?

For example, I'd like to ask python-build to create its isolated build environments inside .local/tmp/python-build inside my project.

Sorry if this is documented somewhere and I didn't see it.

gwerbin avatar Aug 15 '23 13:08 gwerbin

What is the reason behind this request? Is it so you can use the environment for other purposes? What are they?

uranusjr avatar Aug 16 '23 01:08 uranusjr

Related discussion: https://github.com/pypa/pip/issues/12461. A workaround could be to use the standard env vars TEMP, TMP, TMPDIR.

webknjaz avatar Jan 09 '24 17:01 webknjaz

@uranusjr I don't recall the original impetus for my request. I think I was having trouble with filesystem permissions in a restricted environment. It would also be useful in a variety of scenarios like inspecting the results of a failed build.

I think I had tried changing TMPDIR and found that it had no effect, but this was months ago and I don't actually remember. If TMPDIR is meant to work for this purpose, that would probably be fine for anything I'd need.

gwerbin avatar Jan 10 '24 03:01 gwerbin

FWIW we're running into another use case for this, which is when the build process involves complex compilation steps and we would like them cached. We use sccache for this purpose, but if the paths to files changes sccache will view that as a new file and every compilation will be a cache miss. That means that we cannot rely on caching the build results of any artifacts that includes objects from the build environment in the compile line because the path to that environment is not stable.

vyasr avatar Oct 14 '24 17:10 vyasr

Is that because of where the sdist is expanded (#614) or do you actually need to control the location of a build's venv?

layday avatar Oct 14 '24 19:10 layday

For compiled use case (like scikit-build-core and meson-python), it's the build venv.

henryiii avatar Oct 14 '24 19:10 henryiii

@layday I think some envs may have restricted/read-only file system with only a small portion being read-write. This is often the case with containers.

webknjaz avatar Oct 14 '24 19:10 webknjaz

Yes, as Henry says it is the build environment for compiled use cases.

vyasr avatar Oct 14 '24 19:10 vyasr