joystream
joystream copied to clipboard
Improve concurrent downloads mechanism
Colossus has an implementation for queuing and processing tasks in a work stack. There are several problems with the implementation especially the way it is used in the syncing worker. To get the best concurrency the sync worker should add all tasks to the work queue.
We should improve it to get better concurrency without the limitation mentioned above and to work better when we improve on how we fetch tasks in batches (see: https://github.com/Joystream/joystream/issues/5051)