Webconverger
xorriso: "Malformed environment variable SOURCE_DATE_EPOCH encountered"
Causing the binary target to fail.
Writing to 'stdio:live-image-i386.hybrid.iso' completed successfully.
xorriso : NOTE : -return_with SORRY 32 triggered by problem severity SORRY
Makefile:40: recipe for target 'binary' failed
make: *** [binary] Error 32
Seems to have no affect on the final ISO, so I guess this message can be ignored ?
That's a generic error message. The one with actual details will be a few lines up.
Thank you Chris. You're right.
Real issue seems to be:
xorriso 1.4.6 : RockRidge filesystem manipulator, libburnia project.
xorriso : SORRY : Malformed environment variable SOURCE_DATE_EPOCH encountered
xorriso : HINT : Unset SOURCE_DATE_EPOCH before starting xorriso or see https://reproducible-builds.org/specs/source-date-epoch/
xorriso : NOTE : Tolerated problem event of severity 'SORRY'
@kaihendry, did you discover what the problem with this issue is exactly? Your fix doesn't seem to match the symptoms exactly. Apart from generating SOURCE_DATE_EPOCH for only the find command and apart from breaking building with sudo, your fix only seems to change the SOURCE_DATE_EPOCH to be generated from the chroot repo instead of the Debian-live-config repo, which I think would change the date, but the problem here seems to be that it is not set at all?
Though, looking at the Dockerfile, it seems you only add the webconverger subdirectory of this repo to docker, which would probably mean that the .git directory of this repo is not available inside Docker, which would indeed cause a problem?
I have created an alternative fix here. I hope this fixes the build problems @usr was having, once he tested this I'll create a PR.
when running the build using the Dockerfile, the .git directory of this repo would not be available, so no date could be determined, which might break the build.
Ah, which was probably setting SOURCE_DATE_EPOCH to the empty string. Which sounds "malformed" :)
Ah, which was probably setting SOURCE_DATE_EPOCH to the empty string. Which sounds "malformed" :)
Yup, I think that's what would happen.
Sorry, I failed to figure out how to get a variable passing around a Makefile. Will test your branch when I get a chance. Thank you for looking into this! :sweat_smile:
Wait, I just tested 9edba83e38723ecceaeacf8e5f213f7ea1a179dd and I am getting different sha1sums for the resulting ISOs using https://github.com/Webconverger/Debian-Live-config/blob/master/build.sh ... am I missing something?
I haven't actually confirmed reproducible builds (since I don't really care about that), but it the old export SOURCE_DATE_EPOCH stuff used to work, I'd think it would work again. Not sure about any of that, though. Perhaps the xorriso version installed by Docker isn't sufficiently patched or sufficiently new after all. To confirm, you could try reverting 40c7de8a85ca992a3f341d9e04f32dd7cdb27967 and see if that fixes the reproducible builds (on top of my branch, since without my fixes I think SOURCE_DATE_EPOCH won't be properly forwarded anyway).
Sorry for the delay, I git checkout -b revert 40c7de8a85ca992a3f341d9e04f32dd7cdb279671 and it didn't seem to make stable builds. Probably need to @lamby on the case, since I'm out of my depth / time.
I had a look just now, and it seems there seems to be a timestamp in some rock-ridge related file that's different, diffing gives me this:
$ cmp -l live-image-i386.hybrid.iso foo/live-image-i386.hybrid.iso
74001 71 60
108817 71 60
$ hd foo/live-image-i386.hybrid.iso -s 73000 -n 2000
00011d28 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00011ff8 00 00 00 00 00 00 00 00 45 52 ed 01 0a 54 87 01 |........ER...T..|
00012008 52 52 49 50 5f 31 39 39 31 41 54 48 45 20 52 4f |RRIP_1991ATHE RO|
00012018 43 4b 20 52 49 44 47 45 20 49 4e 54 45 52 43 48 |CK RIDGE INTERCH|
00012028 41 4e 47 45 20 50 52 4f 54 4f 43 4f 4c 20 50 52 |ANGE PROTOCOL PR|
00012038 4f 56 49 44 45 53 20 53 55 50 50 4f 52 54 20 46 |OVIDES SUPPORT F|
00012048 4f 52 20 50 4f 53 49 58 20 46 49 4c 45 20 53 59 |OR POSIX FILE SY|
00012058 53 54 45 4d 20 53 45 4d 41 4e 54 49 43 53 50 4c |STEM SEMANTICSPL|
00012068 45 41 53 45 20 43 4f 4e 54 41 43 54 20 44 49 53 |EASE CONTACT DIS|
00012078 43 20 50 55 42 4c 49 53 48 45 52 20 46 4f 52 20 |C PUBLISHER FOR |
00012088 53 50 45 43 49 46 49 43 41 54 49 4f 4e 20 53 4f |SPECIFICATION SO|
00012098 55 52 43 45 2e 20 20 53 45 45 20 50 55 42 4c 49 |URCE. SEE PUBLI|
000120a8 53 48 45 52 20 49 44 45 4e 54 49 46 49 45 52 20 |SHER IDENTIFIER |
000120b8 49 4e 20 50 52 49 4d 41 52 59 20 56 4f 4c 55 4d |IN PRIMARY VOLUM|
000120c8 45 20 44 45 53 43 52 49 50 54 4f 52 20 46 4f 52 |E DESCRIPTOR FOR|
000120d8 20 43 4f 4e 54 41 43 54 20 49 4e 46 4f 52 4d 41 | CONTACT INFORMA|
000120e8 54 49 4f 4e 2e 41 4c 2f 01 00 00 03 04 64 69 00 |TION.AL/.....di.|
000120f8 07 02 fe 01 03 0d e0 0c 00 03 04 73 74 00 0a 31 |...........st..1|
00012108 35 30 39 30 30 38 35 30 30 00 03 04 6e 74 00 04 |509008500...nt..|
00012118 01 01 01 ff 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00012128 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
000124f8
$ hd foo/live-image-i386.hybrid.iso -s 108000 -n 2000
0001a5e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
0001a800 45 52 ed 01 0a 54 87 01 52 52 49 50 5f 31 39 39 |ER...T..RRIP_199|
0001a810 31 41 54 48 45 20 52 4f 43 4b 20 52 49 44 47 45 |1ATHE ROCK RIDGE|
0001a820 20 49 4e 54 45 52 43 48 41 4e 47 45 20 50 52 4f | INTERCHANGE PRO|
0001a830 54 4f 43 4f 4c 20 50 52 4f 56 49 44 45 53 20 53 |TOCOL PROVIDES S|
0001a840 55 50 50 4f 52 54 20 46 4f 52 20 50 4f 53 49 58 |UPPORT FOR POSIX|
0001a850 20 46 49 4c 45 20 53 59 53 54 45 4d 20 53 45 4d | FILE SYSTEM SEM|
0001a860 41 4e 54 49 43 53 50 4c 45 41 53 45 20 43 4f 4e |ANTICSPLEASE CON|
0001a870 54 41 43 54 20 44 49 53 43 20 50 55 42 4c 49 53 |TACT DISC PUBLIS|
0001a880 48 45 52 20 46 4f 52 20 53 50 45 43 49 46 49 43 |HER FOR SPECIFIC|
0001a890 41 54 49 4f 4e 20 53 4f 55 52 43 45 2e 20 20 53 |ATION SOURCE. S|
0001a8a0 45 45 20 50 55 42 4c 49 53 48 45 52 20 49 44 45 |EE PUBLISHER IDE|
0001a8b0 4e 54 49 46 49 45 52 20 49 4e 20 50 52 49 4d 41 |NTIFIER IN PRIMA|
0001a8c0 52 59 20 56 4f 4c 55 4d 45 20 44 45 53 43 52 49 |RY VOLUME DESCRI|
0001a8d0 50 54 4f 52 20 46 4f 52 20 43 4f 4e 54 41 43 54 |PTOR FOR CONTACT|
0001a8e0 20 49 4e 46 4f 52 4d 41 54 49 4f 4e 2e 41 4c 2f | INFORMATION.AL/|
0001a8f0 01 00 00 03 04 64 69 00 07 02 fe 01 03 0d e0 0c |.....di.........|
0001a900 00 03 04 73 74 00 0a 31 35 30 39 30 30 38 35 30 |...st..150900850|
0001a910 30 00 03 04 6e 74 00 04 01 01 01 ff 00 00 00 00 |0...nt..........|
0001a920 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
0001adb0
Note that this just dumps one iso for some context, the difference is in the timestamp (1509008500). @lamby, perhaps this rings a bell for you?
In any case, I confirmed that the SOURCE_DATE_EPOCH is correctly passed to xorriso, with and without my changes, and the same reproducibility problem exists in the current master branch, so I think my changes are not introducing this problem. I'll go ahead and submit them as a PR to be merged (so at least builds as non-root work again) and if I have some leftover time, look at the reproducibility issue more closely.
I also noticed once that git created pack files with different filenames (hashes), but perhaps I was doing something differently then (I could not reproduce this anymore, with or without my changes).
Are you sure you are using the patched xorisso?
I removed the patches and backports-from-sid in 40c7de8a85ca992a3f341d9e04f32dd7cdb27967. At that time, there were no xorriso patches left in this repository, but it still backported xorriso from sid, so I assumed that the sid version has the necessary patches. At that time, the xorriso version (and other backported packages) in sid was the same as in stretch, so it seemed I could safely remove this backporting.
The changelog for the stretch version (1.4.6-1, which I am running) has:
libisoburn (1.4.6-1) unstable; urgency=low
* New upstream release
(...)
+ New environment variable SOURCE_DATE_EPOCH
Which I think is the patch you are referring to?
I also tried upgrading xorriso and its dependencies (libburn4, libisoburn1 and libisofs6) to the sid versions, but that did not make any difference.
Actually, it seems that with the sid versions of these packages, I get a lot more differences in the resulting iso files (around 50).
I get a lot more differences in the resulting iso files (around 50).
Isn't that expected, ie. you are missing the patches that I added to some WebConverger repo?
(This was well over a year ago, sorry if my memory is hazy... plus slammed at the moment for time!)
@lamby, it seems you removed those patches here (https://github.com/Webconverger/Debian-Live-config/commit/75ebaabdf456f8a1b19b358aaf76b8725b5b763d), instead using the sid version, so I assumed the patches were merged. You mention 1.4.6 in the commit message, which I think is also what I was using before, so apparently that version is not completely ok yet (or perhaps some new issue popped up)?
Quite possible. But I'm sorry but I just simply can't remember and don't have the bandwidth/time to reload/try all this out right now…
That's fine, I'm not expecting you to :-) Just trying to get as much info in this ticket for when someone does have time to investigate further (I'm also a bit busy with other stuff right now).