zsync support (delta content sync)

[raffis]

Description

Is: Currently the whole file must be downloaded by clients if something changes on the server (version increase, different checksum). For example one puts a 5G iso file on balloon and changes something small. A balloon client must download the entire 5G again despite many blocks being still the same.

Should: zsync is basically rsync over http. A .zsync metadata file is required which contains blocks and a checksum for each block. A client can download the zsync metadata and determine what block changed and only download those blocks via http range requests (already supported by the balloon api v2).

See: https://github.com/cph6/zsync/tree/master/c

The server must create zsync metadata while a file gets updated (or created). A file may than be fetched via zsync command or balloon libraries which are aware of zsync. This will increase performance alot for big files. (There is not much benefit for small files since the overhead may be bigger than the file itself).

(owncloud has implemented smth similar, but they can use the existing c library (or a fork of it owncloud/zsync), balloon libraries like balloon-node-sync may need a new nodejs module with zsync support).

balloon server version

v2.4.2

PHP version

Operating system

Webserver

[raffis]

Note that .zsync metadata is binary while one the header data is ascii:

  1 zsync: 0.6.2
  2 Filename: MellowPlayer-x86_64.AppImage
  3 MTime: Fri, 08 Feb 2019 07:55:01 +0000
  4 Blocksize: 4096
  5 Length: 100818632
  6 Hash-Lengths: 2,2,5
  7 URL: MellowPlayer-x86_64.AppImage
  8 SHA-1: 401905e39329f0e8373b248b36b5c21038a30fb2

...BINARY....