frozenlist icon indicating copy to clipboard operation
frozenlist copied to clipboard

PEP517 backend is non-deterministic

Open nanonyme opened this issue 2 years ago • 23 comments

Long story short

We noticed in repro test that this package does not result in deterministic output. The problem is creation of temporary directory here https://github.com/aio-libs/frozenlist/blob/8c15ec9/packaging/pep517_backend/_backend.py#L199. It leaks into debuginfo (we spotted this through analysing _frozenlist.cpython-312-aarch64-linux-gnu.so.debug)

Expected behaviour

Debuginfo is deterministic

Actual behaviour

Debuginfo is not deterministic

Steps to reproduce

Build twice, extract debuginfo, run diffoscope on data.

Your environment

freedesktop-sdk.

nanonyme avatar Jan 28 '24 19:01 nanonyme

The root cause seems to be https://github.com/aio-libs/frozenlist/blob/8c15ec9/packaging/pep517_backend/_backend.py#L289 where in-place build is forced to false when building wheel. This means the wheels will always be non-deterministic. This is not great because these days everyone is expected to first create a wheel before installing packages. (this is also what pip does by default)

nanonyme avatar Jan 28 '24 20:01 nanonyme

This was asked on Matrix too and I sent that person to Cython, so they filed https://github.com/cython/cython/issues/5949. I'd rather wait for them to figure out an acceptable solution before trying to invent hacks here. The solution will likely involve a PEP 517 config setting for a custom pre-determined path and/or a setting to build in-place.

webknjaz avatar Mar 06 '24 15:03 webknjaz

That sounds a bit odd though. We build a lot of projects with Cython (we build everything with pypa/build and use pypa/installer to install them) and frozenlist is the only one that is not deterministic.

nanonyme avatar Mar 21 '24 07:03 nanonyme

Iirc most projects here do not create random dirs but build in subdir under source tree.

nanonyme avatar Mar 21 '24 07:03 nanonyme