exo icon indicating copy to clipboard operation
exo copied to clipboard

P2P Download

Open AlexCheema opened this issue 1 year ago • 0 comments

With the new shard download, we have Content-Range resumable downloads with integrity checks so we should be able to give a list of candidate download URLs (in list of priority) and exo can try downloading from each one in turn.

This should be an easy way to get P2P downloads working - each node exposes a HTTP endpoint to serve models to others with Content-Range support. Should just work.

we probably want to keep downloading from the same URL but periodically (e.g. every 5 secs) check if a better candidate is available. any error means try next candidate. also we would want to make sure we don't have partial writes to the file that can corrupt it - this wouldn't ever lead to a corrupt download since we have an integrity check but it might lead to infinite download loop

AlexCheema avatar Feb 04 '25 12:02 AlexCheema