support for Buffers (keys and values)

[jekor]

trafficstars

How much work would it be to add support for Buffers? I have 128-bit keys and would expect 37.5% efficiency encoding their base64 representation to UTF-16 (and don't have a base16k implementation).

I'd be happy to try it myself but would appreciate some warning if most of the code would require rewriting. Thank you.

[mmomtchev]

If you have only Buffer-keys, not much, no algorithms need changing at all (especially if they are fixed-size)

• You will have to rewrite _decodeKey which should return a Buffer from a bucket position
• You need your own _match which will be almost the same but without the charCodeAt conversion
• You need your own _write without the charCodeAt for the keys
• And, optionally, you will probably want to change the way META.keySize is handled, to not have to pass a length value measured in UTF-16 codepoints

Globally, it should be also much faster, since Node can handle the conversion from Uint16 to Buffer internally using only the C++ code. Instead of copying element per element, you can do

_decodeKey(pos) {
    return this.keysData.slice(pos * this.meta[META.keySize],
        (pos+1) * this.meta[META.keySize)).buffer;
}

The Uint16Array internal buffer property is what you need. You have to use .slice, uou can't use the faster .subarray because you need to copy the subarray, otherwise .buffer will return the whole array.

If you need both Buffer-keys and String-keys, you will need an extra field to recognize them and some extra code to return either Buffer either String

[jekor]

Thanks. I'll give it a try.

[jtenner]

Was there any work done on this? I have a need for a shared hashmap that uses buffers for keys and values of fixed size.