js-multiformats
js-multiformats copied to clipboard
Proposal: StreamMultihasher interface
Rational
Current MultihashHasher
API is really simple and a great fit for most cases, however it has certain drawbacks that make it a bad fit for more low-level use cases.
- API returns a
Digest
view for which you always have to allocate a buffer. If you have destination buffer for the multihash like a CAR or even a CID it creates unnecessary overhead of first allocating then copying the bytes into your destination.- It is also worth calling out that web crypto APIs require you to pass a ArrayBufferView.
- API assumes that you can hold complete payload in memory. When working on a large payloads this becomes a real problem. Unfortunately currently browsers do not provide streaming crypto APIs, but there seems to be some proposals to address this, never the less it would still be good to have it in node and non web native implementations.
- Sometimes you need a digest without the multiformat prefix and addressing above two without a way to omit the prefix would defeat the purpose in those instances.
Proposal
I end up defining separate interface to address above limitations and implemented two hashers that makes use of it
I propose we uplift interface used by those implementations, possibly after iterating on it a bit here. I'm happy to update those implementations to align with interface that gets merged here.
Instead of writing up an interface here, I've created https://github.com/multiformats/js-multiformats/pull/261 so it's easier to provide feedback inline and iterate