helia
helia copied to clipboard
Option to run @helia/http as Rapide
After we get #289 completed. We should look into a version/option to run as Rapide.
I'll need to spend a bit more time thinking about this and familiarizing myself with Rapide, but initial thoughts are that each gateways gets a service worker (worker thread if Nodejs). I'm assuming there is some library that provides an api which will let us treat them the same.
This should be unblocked now - an RC of @helia/http is available - npm i @helia/http@next
We currently do round-robin based on a reliability function in our trustless gateway block-brokers. I'm sure we could benefit from some smarter downloading logic.
Making that functionality extensible, so users could implement their own download prioritization/parallelization, would be really cool.
Looks like some related issues will be helpful if we decide to implement some sort of Rapide in Helia:
- https://github.com/ipfs/go-libipfs-rapide/issues/12
- https://www.youtube.com/watch?v=Cv01ePa0G58
Also related tangentially:
- https://github.com/ipfs/specs/issues/444
- https://github.com/ipfs/specs/pull/445
These came about after conversations between @lidel and @Jorropo to
minimize the number of requests Rapide has to do to learn about the shape of the DAG + be able to safely leverage deserialized responses from legacy/dumb gateways and http mirrors as webseeds