graph-node
graph-node copied to clipboard
IPFS: Stop doing files/stat calls, use only cat
Right now we rely on the files/stat ipfs route to fetch file sizes before processing them. However in practice we've found that this API tends to error when cat still succeeds, and generally the more exotic IPFS API methods aren't super well documented or necessarily stable across Kubo versions. It would be more robust to rely on cat as our sole interface to IPFS.
Of course files/stat is serving a purpose here of preventing us of trying to download huge files, but we can achieve the same with cat, because we can cat the files in chunks and stop if we've passed our file size threshold.