Funding proposition of work

[tlaurion]

Hello @taskset,

This is to address https://github.com/tasket/wyng-backup/issues/64#issuecomment-624181519.

Some blocking issues are stopping this project to gain popularity and users. I see:

• #18 #59 #53 but also others you may need to prioritize for QubesOS inclusion, where QubesOS requires encryption prior of integration following their own requirements.

Insurgo would be interested in funding some required work from opencollective fund following original plan for which first transfer has finally occured to Insurgo Initiative.

I would love for you to add a tag under github project, and tag related issues with a "Help needed" or "Fund needed" that would open the door to how funding generally work for big projects to ease funding/collaborative developments for tasks you feel confortable being funded for completion/collaboration:

• Milestones (tasks needed to close a ticket) with funds that would be necessary to accomplish work by community developer.
• Discussion directly in tickets on what Insurgo (and others funders) would commit to fund.
• Payment upon proof of work of milestone, tasks.

Is that something you would be comfortable agreeing?

This is to continue AccessibleSecurity NlNet funded project. This would also targets QubesOS @marmarek and Heads @osresearch to entice community to be aware not only of needed amount of work and cost of work transparency, but also enticing people willing to contribute monetarily/programmatically for food to make things go forward faster for what is considered important by the cabal of the projects themselves, keeping direction and veto.

My goal here is of course to make #54 happen for the present project.

[tasket]

Sounds pretty good. I'm looking at the opencollective and nlnet stuff now...

[tlaurion]

@tasket : New priorities would be:

• [x] Labeling of manual backup session. (Ex: Untouched QubesOS template) to be seen in list SpecificVM and exluded of in pruning of session below
• [x] prune having the possibility to prune all sessions excluding labeled sessions (Prune of all automatic cron sessions but not labeled ones, automatically. End goal here would be to keep labeled "Untouched QubesOS original TemplateVM", and then "Insurgo modified TemplateVM" for users to be able to compare changes. And then "Monthly QubeSOS upgrades".
• [x] Minimizing throughput on verify/send/receive so that network backups sessions are read/written as minimally as possible for slow network environment/expensive mobile networks, while still being able to emergency restore (receive) delta from network. (dom0 will be able to restore everything but itself. Heads will be able to restore root-autosnap to root)
• [ ] QubesOS metadata (qubes.xml resulting of commands to rerun in background/to reapply QubesOS properties on restore (save on send)
• [x] Make sure that shell version of the script is compatible for receive and verify

[tasket]

@tlaurion

On minimizing consumption of network bandwidth...

Wyng is already 80% there, however I can give you one more big "leg up" in net efficiency in the short term by making send_volume() internally remap-compatible (currently remap is the job of receive_volume() so I'm proposing a way to remap in the opposite direction which results in zero net access).

Further improvement is possible by implementing things like a clone volume command (direct cloning of a volume in archive without going through dedup), or improving dedup efficiency.

---

On the shell script...

For a new iteration of the archive format with encryption, that would be a major or impossible hurdle for the shell script. I really don't know what encryption tools are available in heads/busybox environments, but even if available getting them to batch process files and format like Python libraries is a very big "if".

Without encryption, it should still be possible to access future archives from shell scripts, and possibly even with cryptographic verification. I understand that what you're doing may not need encryption, and that you're already performing a kind of crypto verification on the wyng metadata.

---

Qubes metadata...

At first this will be a save-and-retrieve only feature using a staging area. User/admin will have to script the re-application of metadata to Qubes OS themselves.

---

Automatic pruning and labeling...

Very do-able and I could even post as v0.2 'testing' feature.

[tasket]

@tlaurion To get things going, here is a preliminary list of the individual issues with requested funding:

$75 - issue #59 - record comments

$150 - issue #71 - prune by comments

$500 - issues #24 and #39 - network efficiency using differential modes

Let me know what you think or if you need to make a counter-offer.

[tasket]

BTW, I am including issue #70 in with 24 and 39.

The recent commits for the 'autoprune' feature will also provide the framework for implementing 71.

[tlaurion]

@tasket : Agreeing with the numbers here. Goal is pay upon proof of work, where opencollective invoice should ideally be linked to an issue to be resolved, facilitating tracking of work being funded by the collective upon payment of validated PoW.