node-xxhash icon indicating copy to clipboard operation
node-xxhash copied to clipboard

different results from java

Open vast-z opened this issue 5 years ago • 8 comments

java code: long key = LongHashFunction.xx().hashChar('a'); String key = Long.toHexString(key); System.out.println(key);

result: e513e02c99167f96

js code: const xxhash = require('xxhash') const buffer = Buffer.from('a') const key = xxhash.hash64(buffer, 0).toString('hex') console.log(key)

result: 5b6e8ca9f1c44ed2

I don't know the reason, thacks for the author!!!

vast-z avatar Sep 11 '19 05:09 vast-z

I don't know, but it could be that the Java library you're using is based on a different version/configuration/implementation of xxhash that produces different results? This project uses xxhash's C implementation directly, so unless the Java library is also a binding directly to the C implementation, I would think this module would be more "correct."

mscdex avatar Sep 11 '19 06:09 mscdex

I agree with u, thank you very much. I found these lib (java/js) in xxhash web page, but different results bothered me.

vast-z avatar Sep 11 '19 06:09 vast-z

https://cyan4973.github.io/xxHash/ java: https://github.com/OpenHFT/Zero-Allocation-Hashing js(node): https://npmjs.org/package/xxhash

vast-z avatar Sep 11 '19 06:09 vast-z

Looking at the Java implementation I see the difference: they are converting the character to a two-byte value before hashing and they are returning the value reversed for some reason (although both implementations should be returning little endian always).

To reproduce the Java result, you'd have to do something like this:

const buffer = Buffer.from([0x61, 0x00]); 
const key = xxh.hash64(buffer, 0).swap64().toString('hex');

mscdex avatar Sep 11 '19 06:09 mscdex

thank you very much! You are so kind.I'll try that later…I was in a hurry yesterday…I made node call java finally…😄

vast-z avatar Sep 12 '19 01:09 vast-z

Same for python: https://pypi.org/project/xxhash/

felixhao28 avatar Nov 04 '19 03:11 felixhao28

Same for 32bit hash results: Java sample hash: f28bd32f (1111 0010 1000 1011 1101 0011 0010 1111)

JS sample hash decoded using big endian xxHash.hash(buffer, seed,'hex') 2fd38bf2 (0010 1111 1101 0011 1000 1011 1111 0010) JS sample hash decoded using little endian xxHash.hash(buffer, seed).toString(16) f28bd32f (1111 0010 1000 1011 1101 0011 0010 1111)

wrobelma avatar Nov 24 '19 11:11 wrobelma

Was switching JS libraries and found out that all other (js implementations) are returning reversed order. Don't know what is correct way. But had to do small inversion function to make it same output.

const xxhashOutput = require('xxhash').hash64(Buffer.from('text', 'utf8'), 0xcafebabe).toString('hex')
const xxhashjsOutput = require('xxhashjs').h64('text', 0xcafebabe).toString(16)
const inverted = '0'.repeat(16 - xxhashjsOutput.length) + xxhashjsOutput // it is missing leading 0s

let flipped = ''
while (flipped.length < inverted.length) {
  flipped = flipped + inverted.substr(inverted.length - flipped.length - 2, 2)
}

// flipped === xxhashOutput
console.log(flipped, xxhashOutput)

jakubriedl avatar Oct 30 '20 09:10 jakubriedl