jemalloc-sys is not reproducible

[nagisa]

Building project with jemalloc-sys as a dependency results in a closure that's not entirely reproducible. In particular, the build output will install a large number of other unnecessary files, including intermediate build artifacts. In particular instance of jemalloc-sys there will be just one different file: config.log:

Diff

--- /nix/store/w7asdvykp8qnq27rd1xz0rip97rbyhyj-rust_jemalloc-sys-0.3.2-lib/lib/jemalloc-sys.out/build/config.log	1970-01-01 03:00:01.000000000 +0300
+++ /nix/store/w7asdvykp8qnq27rd1xz0rip97rbyhyj-rust_jemalloc-sys-0.3.2-lib.check/lib/jemalloc-sys.out/build/config.log	1970-01-01 03:00:01.000000000 +0300
@@ -508,7 +508,7 @@
 conftest.c:48:6: warning: conflicting types for built-in function 'log' [-Wbuiltin-declaration-mismatch]
  char log ();
       ^~~
-/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: /build/ccve8EIc.o: in function `main':
+/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: /build/ccwdPg4g.o: in function `main':
 /build/jemalloc-sys-0.3.2/target/build/jemalloc-sys.out/build/conftest.c:52: undefined reference to `log'
 collect2: error: ld returned 1 exit status
 configure:7639: $? = 1
@@ -1009,7 +1009,7 @@
 configure:10328: result: yes
 configure:10333: checking for dlsym
 configure:10333: gcc -o conftest -std=gnu11 -Wall -Wsign-compare -Wundef -Wno-format-zero-length -pipe -g3 -fvisibility=hidden -O3 -funroll-loops -O3 -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -Wall -O3 -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -Wall -D_GNU_SOURCE -O3 -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -Wall conftest.c -lm  -lpthread >&5
-/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: /build/ccqJOSJ9.o: in function `main':
+/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: /build/ccCCuODv.o: in function `main':
 /build/jemalloc-sys-0.3.2/target/build/jemalloc-sys.out/build/conftest.c:100: undefined reference to `dlsym'
 collect2: error: ld returned 1 exit status
 configure:10333: $? = 1
@@ -1255,7 +1255,7 @@
 configure:10915: result: yes
 configure:10928: checking for issetugid
 configure:10928: gcc -o conftest -std=gnu11 -Wall -Wsign-compare -Wundef -Wno-format-zero-length -pipe -g3 -fvisibility=hidden -O3 -funroll-loops -O3 -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -Wall -O3 -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -Wall -D_GNU_SOURCE -D_REENTRANT -O3 -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -Wall conftest.c -lm  -lpthread -ldl >&5
-/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: /build/ccIa2k8n.o: in function `main':
+/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: /build/ccWVvm7H.o: in function `main':
 /build/jemalloc-sys-0.3.2/target/build/jemalloc-sys.out/build/conftest.c:109: undefined reference to `issetugid'
 collect2: error: ld returned 1 exit status
 configure:10928: $? = 1
@@ -1375,7 +1375,7 @@
 configure:10928: result: no
 configure:10941: checking for _malloc_thread_cleanup
 configure:10941: gcc -o conftest -std=gnu11 -Wall -Wsign-compare -Wundef -Wno-format-zero-length -pipe -g3 -fvisibility=hidden -O3 -funroll-loops -O3 -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -Wall -O3 -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -Wall -D_GNU_SOURCE -D_REENTRANT -O3 -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -Wall conftest.c -lm  -lpthread -ldl >&5
-/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: /build/cctEmE6r.o: in function `main':
+/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: /build/ccRRT9HM.o: in function `main':
 /build/jemalloc-sys-0.3.2/target/build/jemalloc-sys.out/build/conftest.c:109: undefined reference to `_malloc_thread_cleanup'
 collect2: error: ld returned 1 exit status
 configure:10941: $? = 1
@@ -1495,7 +1495,7 @@
 configure:10941: result: no
 configure:10956: checking for _pthread_mutex_init_calloc_cb
 configure:10956: gcc -o conftest -std=gnu11 -Wall -Wsign-compare -Wundef -Wno-format-zero-length -pipe -g3 -fvisibility=hidden -O3 -funroll-loops -O3 -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -Wall -O3 -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -Wall -D_GNU_SOURCE -D_REENTRANT -O3 -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -Wall conftest.c -lm  -lpthread -ldl >&5
-/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: /build/ccoDPhwz.o: in function `main':
+/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: /build/ccsdpPIT.o: in function `main':
 /build/jemalloc-sys-0.3.2/target/build/jemalloc-sys.out/build/conftest.c:109: undefined reference to `_pthread_mutex_init_calloc_cb'
 collect2: error: ld returned 1 exit status
 configure:10956: $? = 1

An obvious fix would be to just not install the unnecessary files. Did not yet investigate it is crate2nix or buildRustCrate (I think its going to be latter) that’s installing them, but since bugs here have significantly greater signal-to-noise ratio, reporting it here.

[nagisa]

cc @andir

[andir]

This sounds completly unrelated to both crate2nix and buildRustCrate. The reproducibility issue is probably due to jemallocator or jemalloc-sys bundling their own jemalloc. Try setting JEMALLOC_OVERRIDE to the path to the jemalloc within nixpkgs. That jemalloc version actually reproduces for me on x86_64-linux.

[nagisa]

This sounds completly unrelated to both crate2nix and buildRustCrate.

It is related at least to an extent that they install way too much into the outputs. Once an .rlib/so/etc is produced, nothing in the traditional cargo target/ but that output needs to be retained. As things are now, we retain things that are as menial as configure script output – and that will eventually come to cause issues with more than just jemalloc-sys.

[nagisa]

Another instance of the same: opencv.

Diff

--- /nix/store/20wcnnmsb742lz5p19z71l7nrwna8hqv-rust_opencv-0.29.3-lib/lib/opencv.opt	1970-01-01 03:00:01.000000000 +0300
+++ /nix/store/20wcnnmsb742lz5p19z71l7nrwna8hqv-rust_opencv-0.29.3-lib.check/lib/opencv.opt	1970-01-01 03:00:01.000000000 +0300
@@ -53,8 +53,6 @@
 cargo:rerun-if-env-changed=OPENCV_LINK_LIBS
 cargo:rerun-if-env-changed=OPENCV_LINK_PATHS
 cargo:rerun-if-env-changed=OPENCV_INCLUDE_PATHS
-Generating module 'freetype' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/freetype.hpp
 Generating module 'imgcodecs' from headers:
 	/build/opencv-0.29.3/headers/3.4/opencv2/imgcodecs.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/imgcodecs/imgcodecs.hpp
@@ -84,27 +82,20 @@
 Generating module 'hdf' from headers:
 	/build/opencv-0.29.3/headers/3.4/opencv2/hdf.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/hdf/hdf5.hpp
+Generating module 'bgsegm' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/bgsegm.hpp
 Generating module 'bioinspired' from headers:
 	/build/opencv-0.29.3/headers/3.4/opencv2/bioinspired.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/bioinspired/bioinspired.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/bioinspired/retina.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/bioinspired/retinafasttonemapping.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/bioinspired/transientareassegmentationmodule.hpp
-Generating module 'bgsegm' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/bgsegm.hpp
 Generating module 'highgui' from headers:
 	/build/opencv-0.29.3/headers/3.4/opencv2/highgui.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/highgui/highgui.hpp
 Generating module 'line_descriptor' from headers:
 	/build/opencv-0.29.3/headers/3.4/opencv2/line_descriptor.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/line_descriptor/descriptor.hpp
-Generating module 'photo' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/photo.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/photo/photo.hpp
-Generating module 'aruco' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/aruco/dictionary.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/aruco.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/aruco/charuco.hpp
 Generating module 'face' from headers:
 	/build/opencv-0.29.3/headers/3.4/opencv2/face.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/face/bif.hpp
@@ -116,40 +107,81 @@
 	/build/opencv-0.29.3/headers/3.4/opencv2/face/facerec.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/face/mace.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/face/predict_collector.hpp
+Generating module 'aruco' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/aruco/dictionary.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/aruco.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/aruco/charuco.hpp
+Generating module 'xfeatures2d' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/xfeatures2d.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/xfeatures2d/nonfree.hpp
+Generating module 'xobjdetect' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/xobjdetect.hpp
+Generating module 'superres' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/superres.hpp
+Generating module 'videostab' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/videostab.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/deblurring.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/fast_marching.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/fast_marching_inl.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/frame_source.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/global_motion.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/inpainting.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/log.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/motion_core.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/motion_stabilizing.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/optical_flow.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/outlier_rejection.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/ring_buffer.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/stabilizer.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/wobble_suppression.hpp
+Generating module 'xphoto' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/xphoto.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/xphoto/bm3d_image_denoising.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/xphoto/dct_image_denoising.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/xphoto/inpainting.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/xphoto/tonemap.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/xphoto/white_balance.hpp
+Generating module 'calib3d' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/calib3d.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/calib3d/calib3d.hpp
+Generating module 'viz' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/viz/types.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/viz/widgets.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/viz.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/viz/viz3d.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/viz/vizcore.hpp
+Generating module 'ccalib' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/ccalib.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/ccalib/multicalib.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/ccalib/omnidir.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/ccalib/randpattern.hpp
+Generating module 'freetype' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/freetype.hpp
+Generating module 'photo' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/photo.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/photo/photo.hpp
 Generating module 'fuzzy' from headers:
 	/build/opencv-0.29.3/headers/3.4/opencv2/fuzzy.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/fuzzy/fuzzy_F0_math.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/fuzzy/fuzzy_F1_math.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/fuzzy/fuzzy_image.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/fuzzy/types.hpp
+Generating module 'dpm' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/dpm.hpp
+Generating module 'video' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/video.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/video/background_segm.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/video/tracking.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/video/video.hpp
 Generating module 'phase_unwrapping' from headers:
 	/build/opencv-0.29.3/headers/3.4/opencv2/phase_unwrapping.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/phase_unwrapping/histogramphaseunwrapping.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/phase_unwrapping/phase_unwrapping.hpp
-Generating module 'superres' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/superres.hpp
 Generating module 'plot' from headers:
 	/build/opencv-0.29.3/headers/3.4/opencv2/plot.hpp
-Generating module 'sfm' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/sfm.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/conditioning.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/fundamental.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/io.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/numeric.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/projection.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/reconstruct.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/robust.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/simple_pipeline.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/triangulation.hpp
-Generating module 'calib3d' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/calib3d.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/calib3d/calib3d.hpp
-Generating module 'xobjdetect' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/xobjdetect.hpp
-Generating module 'objdetect' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/objdetect.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/objdetect/detection_based_tracker.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/objdetect/objdetect.hpp
+Generating module 'ml' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/ml.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/ml/ml.hpp
 Generating module 'dnn' from headers:
 	/build/opencv-0.29.3/headers/3.4/opencv2/dnn.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/dnn/all_layers.hpp
@@ -158,20 +190,10 @@
 	/build/opencv-0.29.3/headers/3.4/opencv2/dnn/layer.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/dnn/shape_utils.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/dnn/utils/inference_engine.hpp
-Generating module 'ccalib' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/ccalib.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/ccalib/multicalib.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/ccalib/omnidir.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/ccalib/randpattern.hpp
-Generating module 'dpm' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/dpm.hpp
-Generating module 'xphoto' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/xphoto.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/xphoto/bm3d_image_denoising.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/xphoto/dct_image_denoising.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/xphoto/inpainting.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/xphoto/tonemap.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/xphoto/white_balance.hpp
+Generating module 'objdetect' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/objdetect.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/objdetect/detection_based_tracker.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/objdetect/objdetect.hpp
 Generating module 'shape' from headers:
 	/build/opencv-0.29.3/headers/3.4/opencv2/shape.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/shape/emdL1.hpp
@@ -179,46 +201,24 @@
 	/build/opencv-0.29.3/headers/3.4/opencv2/shape/shape.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/shape/shape_distance.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/shape/shape_transformer.hpp
-Generating module 'video' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/video.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/video/background_segm.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/video/tracking.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/video/video.hpp
-Generating module 'xfeatures2d' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/xfeatures2d.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/xfeatures2d/nonfree.hpp
+Generating module 'sfm' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/sfm.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/conditioning.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/fundamental.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/io.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/numeric.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/projection.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/reconstruct.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/robust.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/simple_pipeline.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/sfm/triangulation.hpp
+Generating module 'stitching' from headers:
+	/build/opencv-0.29.3/headers/3.4/opencv2/stitching.hpp
+	/build/opencv-0.29.3/headers/3.4/opencv2/stitching/warpers.hpp
 Generating module 'features2d' from headers:
 	/build/opencv-0.29.3/headers/3.4/opencv2/features2d.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/features2d/features2d.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/features2d/hal/interface.h
-Generating module 'videostab' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/videostab.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/deblurring.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/fast_marching.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/fast_marching_inl.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/frame_source.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/global_motion.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/inpainting.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/log.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/motion_core.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/motion_stabilizing.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/optical_flow.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/outlier_rejection.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/ring_buffer.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/stabilizer.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/videostab/wobble_suppression.hpp
-Generating module 'stitching' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/stitching.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/stitching/warpers.hpp
-Generating module 'ml' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/ml.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/ml/ml.hpp
-Generating module 'viz' from headers:
-	/build/opencv-0.29.3/headers/3.4/opencv2/viz/types.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/viz/widgets.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/viz.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/viz/viz3d.hpp
-	/build/opencv-0.29.3/headers/3.4/opencv2/viz/vizcore.hpp
 Generating module 'videoio' from headers:
 	/build/opencv-0.29.3/headers/3.4/opencv2/videoio.hpp
 	/build/opencv-0.29.3/headers/3.4/opencv2/videoio/registry.hpp

Again, the final rlib is bit-for-bit equivalent, but the build.rs script does not have reproducible operation. And so some log that we save ends up being an issue.

This happens even if opencv is built against the native library; the order differs because build.rs uses a HashSet to keep a list of modules.

[kolloch]

If I understand correctly, opencv uses a build script that is non-deterministic in itself but the relevant binary outputs are reproducible. Unfortunately, the log of the build script is also in the outputs and changes between invocations.

1. I think, the best way would be to have the build script fixed. Why? Because if there are any other reproducibility issues, the best way to debug that is if all build steps happen in the same order always.

2. In theory, we could somehow try to make hashmaps reproducible in tests by somehow pinning the hashmap seed etc. This is beyond my interest currently. If you want to have a go, feel free, obviously. I think it would be so cool!

3. Change this code here. a. Put the log into a tmp file. Unfortunately, that also means there is less information for debugging if things go wrong. b. I haven't used this a lot, so not sure if adequate but we could at least separate the log into a separate "output".

[nagisa]

I think, the best way would be to have the build script fixed. Why? Because if there are any other reproducibility issues, the best way to debug that is if all build steps happen in the same order always.

Very broadly, I would agree, but given the size of the ecosystem and lack of enforcement this is going to be a never ending, uphill battle.

A few rustc developers consider turing-complete build scripts to have been a major mistake and we’ve been pondering about alternative options, such as e.g. compiling them as wasm and running them in a special sandboxing wasm interpreter, but there hasn’t been any drive to implement anything of the sort quite yet.

And so, I believe that we should add at least a mode of operation to crate2nix that saves the least amount of possible information (i.e. no logs from build scripts, maybe there’s something else) to reduce the possible non-reproducibility in outputs to the lowest level feasible.

[kolloch]

I agree with either splitting this into a different output or removing it from the output alltogether. I personally will not be working on this, though. I think it should be relatively simple if you know nix.

So this is option 3 above:

3. Change [this code here](https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/rust/build-rust-crate/configure-crate.nix#L137-L155).
   a. Put the log into a tmp file. Unfortunately, that also means there is less information for debugging if things go wrong.
   b. I haven't used this a lot, so not sure if adequate but we could at least separate the log into a separate "output".

I'd appreciate someone contributing that to nixpkgs!