flatpak
Async downloading of modules
Linux distribution and version : Fedora 34
Flatpak-builder version : 1.0.12
Flatpak version : 1.11.1
Description of the problem
Flatpak-builder will only download modules one at a time. If one of the modules is downloading slow, this results in taking more time than it would if it could download other modules at the same time.
Steps to reproduce
- flatpak-builder builds manifest.json
- Watch as it downloads modules one at a time
Additional notes
The output of the command would likely need to be more simple to allow the person to see all of the modules at once if there are many. There would likely need to be a check to make sure that the same thing is not being downloaded twice.
What could be parallelized, specifically, is the building and the downloading. Builder could start downloading the sources of the next module(s) while the current one is building. Unfortunately I don't think we can parallelize something beside the sources of the first module when downloading the SDK.
Related: for some sources with http downloads, aria2c was four times faster than wget, and aria2c already supports parallel fetching, which helps a lot when a server is the bottleneck. Unfortunately, libaria2 offers a C++-only API*.
* libaria2 C API: someone asked if a C API was possible 5 years ago, and the answer was yes it would be acceptable, if optional https://github.com/aria2/aria2/issues/584#issuecomment-192587853.
Ah, I think that's a different feature request. I simply want it to download multiple modules at once rather going one by one which should still speed things up initially. Related but not quite the same.
I think the concept of aira doesn't age well. With HTTP/2 and QUIC modern servers expect one connection per client. Modern servers also have less of a struggle saturating a gigabit connection.
Eitherway I think its safe to say that its not worth depending on.
