Trailing zeroes in patched file

[Lupus]

I'm trying to use go-sync as a library and run into random corruption of patched files. Looking at hex diff, pattern is always the same - patched file size gets rounded up to block size and extra space is filled with zeroes:

--- /dev/fd/63  2021-12-23 15:03:49.189605458 +0000
+++ /dev/fd/62  2021-12-23 15:03:49.189605458 +0000
@@ -189527,4 +189527,4 @@
 002e6bd0  bc 35 c4 35 c8 35 cc 35  d0 35 00 00 00 00 00 00  |.5.5.5.5.5......|
 002e6be0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
 *
-00303600
+00304000

Block size that I use is 4096 (0x1000), original file size is 3159552, after patching it ends up as 3162112 - 771.375 blocks vs 772.0 blocks.

I double checked that I pass the correct FileSize and BlockSize. Checksum data is read separately and is 15440 bytes long, which looks sane 5440/(4+16) = 772. BlockCount in summary is set to length of checksums array, returned by chunks.LoadChecksumsFromReader. Then I construct blob source like this:

resolver := blocksources.MakeFileSizedBlockResolver(uint64(summary.GetBlockSize()), summary.GetFileSize())
blockSource := blocksources.NewReadSeekerBlockSource(myReadSeeker, resolver)

Is there something that I'm missing? Glancing at the code MakeFileSizedBlockResolver should do the right thing, and truncate last partial block end offset to be no larger than the file, but somehow those trailing zeroes crip in.

go-sync v0.0.0-20200808161209-d9b3aeb508db

[Redundancy]

It's quite possible that there is a bug. The was at least one test that was failing intermittently likely with a race condition.

Since I can't work on this due to contractual issues, your best bet is to see if there is a maintained fork, or do so yourself.

On Thu, Dec 23, 2021, 10:20 Konstantin A. Olkhovskiy < @.***> wrote:

I'm trying to use go-sync as a library and run into random corruption of patched files. Looking at hex diff, pattern is always the same - patched file size gets rounded up to block size and extra space is filled with zeroes:

--- /dev/fd/63 2021-12-23 15:03:49.189605458 +0000 +++ /dev/fd/62 2021-12-23 15:03:49.189605458 +0000 @@ -189527,4 +189527,4 @@ 002e6bd0 bc 35 c4 35 c8 35 cc 35 d0 35 00 00 00 00 00 00 |.5.5.5.5.5......| 002e6be0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * -00303600 +00304000

Block size that I use is 4096 (0x1000), original file size is 3159552, after patching it ends up as 3162112 - 771.375 blocks vs 772.0 blocks.

I double checked that I pass the correct FileSize and BlockSize. Checksum data is read separately and is 15440 bytes long, which looks sane 5440/(4+16) = 772. BlockCount in summary is set to length of checksums array, returned by chunks.LoadChecksumsFromReader. Then I construct blob source like this:

resolver := blocksources.MakeFileSizedBlockResolver(uint64(summary.GetBlockSize()), summary.GetFileSize())blockSource := blocksources.NewReadSeekerBlockSource(myReadSeeker, resolver)

Is there something that I'm missing? Glancing at the code MakeFileSizedBlockResolver should do the right thing, and truncate last partial block end offset to be no larger than the file, but somehow those trailing zeroes crip in.

go-sync v0.0.0-20200808161209-d9b3aeb508db

— Reply to this email directly, view it on GitHub https://github.com/Redundancy/go-sync/issues/20, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABGJ45GETW57B6XHBOOGIVTUSM43ZANCNFSM5KU7UEAA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>