Only 1.8% usable when updating from Firefox-63.0.3 to 64.0

[probonopd]

To be analyzed. Only 1.8% usable when updating from Firefox-63.0.3 to 64.0 - why?

me@host:~$ sudo Downloads/AppImageUpdate-x86_64.AppImage /isodevice/Applications/Firefox-63.0.3.glibc2.7-x86_64.AppImage 
AppImageUpdate version 1-alpha (commit 95cebb0), build 389 built on 2018-12-02 19:19:03 UTC
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
Updating from Bintray via ZSync
zsync2: Target file: /isodevice/Applications/Firefox-64.0.glibc2.3.4-x86_64.AppImage
zsync2: Reading seed file: /isodevice/Applications/Firefox-63.0.3.glibc2.7-x86_64.AppImage
zsync2: Usable data from seed files: 1.803200%

Files: https://bintray.com/probono/AppImages/Firefox#files

[TheAssassin]

How do you expect anybody to analyze this? Who knows how Firefox built their static binaries? Different machine - different settings?

[ferion11]

I have made some tests using the kernel sources from this link: Kernel5.5 and Kernel5.4 to seed.

• The zsync file with the url inside: linux-5.5.tar.xz.zsync.zip

Not only I got 0% usable, the result file is corrupted (even the zsync2 saying that the checksum is ok). I have used one local http server too, and the same result. Have zsync2 the supports to XZ compressed data?

[probonopd]

Honestly I wrote "to be analyzed" because at the time I had no idea how to go about analyzing this (and still don't have). One would need some tool that could detect duplicated sections that are the same in 2 binary files, kinda diff for binaries. Do you know one? (Running xxd and then diff probably won't work if e.g., bytes end up in different positions in the line)

[ferion11]

I use the wxHexEditor on linux. But I would love to use something like the Windows FlexHEX, look that that :hearts: screenshot :hearts: :

[ferion11]

I have tested now the FlexHEX Version 2.7 and Hex Workshop v6.8 on wine (32bits) and both worked very well. I recommend the use of these two :smile:

[ferion11]

I have made some tests here. If the zsync2 have only raw binary comparison, the results will be the same:

• good results: zip (.zip), LZMA2 (.7z), tar archive (.tar), RAR v5 (.rar)
• bad results: gzip (.tar.gz), bzip2 (.tar.bz2), XZ (*.tar.xz)

[TheAssassin]

For the zsync algorithm (like for the original rsync algorithm) you need so-called block-aligned archives. SquashFS for instance can be configured in a way that works well.

.7z should work all the time, no matter how it's compressed. You can also use LZMA version 1 just fine. For .zip I'm not entirely sure if they do block alignment.

None of the formats you listed as bad can provide that, because they're basically compressed streams of tar archives. The compression algorithm cannot know about the internal structure. The ones you listed as good compress with the file layout in mind, they don't just concatenate the files but align the beginnings in a special way, inserting some padding before appending a new file. This way, if I split the archive in blocks (like zsync does), same files will create same blocks.

Uncompressed .tar won't work either by the way, because .tar doesn't do any alignment either.

ISO files by the way should work just fine, as ISO does block alignment.

[ferion11]

Uncompressed .tar won't work either by the way, because .tar doesn't do any alignment either.

Uncompressed .tar should work, the tar archive (.tar) is in the list of good results (it have alignment).

[TheAssassin]

Hm... I thought it hadn't, but if it has, that's good.

[ferion11]

Just for curiosity: gzip (.tar.gz) using the gzip option --rsyncable have good results too.

[lslvr]

How do you expect anybody to analyze this? Who knows how Firefox built their static binaries? Different machine - different settings?

Well, instead of diffing binary files, one could compare results against the original zsync.