helia icon indicating copy to clipboard operation
helia copied to clipboard

Option to run @helia/http as Rapide

Open meandavejustice opened this issue 1 year ago • 3 comments

After we get #289 completed. We should look into a version/option to run as Rapide.

I'll need to spend a bit more time thinking about this and familiarizing myself with Rapide, but initial thoughts are that each gateways gets a service worker (worker thread if Nodejs). I'm assuming there is some library that provides an api which will let us treat them the same.

meandavejustice avatar Dec 11 '23 22:12 meandavejustice

This should be unblocked now - an RC of @helia/http is available - npm i @helia/http@next

achingbrain avatar Jan 17 '24 13:01 achingbrain

We currently do round-robin based on a reliability function in our trustless gateway block-brokers. I'm sure we could benefit from some smarter downloading logic.

Making that functionality extensible, so users could implement their own download prioritization/parallelization, would be really cool.

Looks like some related issues will be helpful if we decide to implement some sort of Rapide in Helia:

  • https://github.com/ipfs/go-libipfs-rapide/issues/12
  • https://www.youtube.com/watch?v=Cv01ePa0G58

SgtPooki avatar Aug 15 '24 16:08 SgtPooki

Also related tangentially:

  • https://github.com/ipfs/specs/issues/444
  • https://github.com/ipfs/specs/pull/445

These came about after conversations between @lidel and @Jorropo to

minimize the number of requests Rapide has to do to learn about the shape of the DAG + be able to safely leverage deserialized responses from legacy/dumb gateways and http mirrors as webseeds

SgtPooki avatar Aug 19 '24 16:08 SgtPooki