CAR format support 🌐✨🚘

[olizilla]

Being able to fetch content as a CAR from nftstorage.link would be rad. It'd allow folks to fetch and verify content themselves, with the nftstorage.link speed boost.

We need it for the web3.storage client.get method. We could hit the nftstorage.link endpoint directly from that rather than via the api worker. We're getting reports of larger files failing to fetch via GET /cid/:cid which is currently just a proxy for ipfs.io. e.g https://github.com/web3-storage/web3.storage/issues/840

The implementation should just use the new /ipfs/:cid?format=car search param that landed in go-ipfs v0.13. If other gateways take a while to adopt this we should implement gets as hitting ipfs.io only, so at least we start to benefit from the cdn / caching behavioiur for them in the meantime.

[olizilla]

right now it's often more reliable to do a

dagula get <cid I got from w3>

than it is to do a

w3 get <cid I got from w3>

because the current implementation of web3.storage.get() is a proxy for the dweb.link gateway. The content may not be on those machines, and they get very busy with public use.

what if we add /ipfs/:cid?format=car support by only racing upstreams that support CARs along with adding a new race participant as a dagula client that fetches the blocks from elastic-ipfs...

[Jorropo]

I just want to link the relevent spec as well as mention that Accept: application/vnd.ipld.car is the way we expect applications to use this feature (this is because thx to HTTP2 this allows to only send the header once). https://github.com/ipfs/specs/blob/main/http-gateways/TRUSTLESS_GATEWAY.md#accept-request-header