Replicated service and reproducible patch archives

[sarum9in]

We can host multiple instances of the service provided that they are able to catch up with the main replica on their own. For this we need to be able to generate exactly same zip files

• https://wiki.debian.org/ReproducibleBuilds/TimestampsInZip
• We can use revision commit timestamp for all files, thanks @textshell for suggestion

Snippet of python code how we can do it:

import zipfile

with zipfile.ZipFile('/tmp/test.zip', 'w') as z:
    f = zipfile.ZipInfo()
    f.filename = 'hello.txt'
    f.date_time = (1980, 1, 1, 0, 0, 0)
    f.compress_type = zipfile.ZIP_DEFLATED
    z.writestr(f, 'hello world')
    f = zipfile.ZipInfo()
    f.filename = 'world.txt'
    f.date_time = (1980, 1, 1, 0, 0, 0)
    f.compress_type = zipfile.ZIP_DEFLATED
    z.writestr(f, 'hello big world')

[sarum9in]

Alternative is to push patch zips to github repos as releases and generate links to them in wrapfile that project acquires. This splits wrap creation and wrap hosting paths into 2 separate things.

• Serving path: use github directly, rely on release feature.
• Maintenance path: use wrapdb server.

[sarum9in]

As @jpakkane noticed this has a disadvantage of tying wrapdb to github. The way to solve it is to proxy get_zip calls to github, the service itself is stateless and we can switch to any other backing store anytime we want. Moreover we don't even have to change old wraps that way.

[tp-m]

It seems to me that currently the weak link is the wrapdb server, so proxying the get_zip call wouldn't really solve the main problem that prevents me from using wrapdb wraps today.

[sarum9in]

The idea to address this is to run multiple read-only mirrors with eventual consistency. Meson should be able to select a working one from the pool (TBD how).