php-xxhash icon indicating copy to clipboard operation
php-xxhash copied to clipboard

Adding support to the 64bit version of xxhash

Open nheimann1 opened this issue 9 years ago • 5 comments

I needed the 64 bit version, I assumed other people will need it too. The rest of the details are in the README.md file.

General note: Since PHP doesn't support unsigned long values, the xxhash64() function can return negative values (this is the same case with the JAVA library). In case you are working in multiple platforms you can use the xxhash64Unsigned() version I added, or casting the results in other platforms from unsigned long to long

nheimann1 avatar Sep 20 '15 14:09 nheimann1

Thank you for adding this! Was looking for a 64-bit implementation.

arcataroger avatar Oct 14 '15 20:10 arcataroger

I'd like to have a php function call for createing a xxhash from a file not from a string. To handle with strings is very poor for very large data. Like md5_file ().

Thanks a lot.

GeThi255

GeThi255 avatar Nov 05 '16 21:11 GeThi255

Can you do xxhash(file_get_contents('file'))? At work we do this with PDFs up to 20 MB or so and it works fine.

arcataroger avatar Nov 05 '16 21:11 arcataroger

Thank you very much for the solution. But this kills the performance.

1st try: $xxhash = xxhash32(file_get_contents($file));
1.32 second for 1.4 GB

2nd try: $xxhash = xxhash64(file_get_contents($file)); 1.08 second for 1.4 GB (same file)

3rd try: $xxhash = exec ('/usr/local/bin/xxh64sum ' . escapeshellarg($file) . ' 2> /dev/null', $RetVal, $RetVar); 0.34 second for 1.4 GB (same file)

My largest file has 143 GB. It's a VM image.

GeThi255 avatar Nov 05 '16 22:11 GeThi255

And the max file size is limited by the RAM.

Tried with 17 GB file: PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted

GeThi255 avatar Nov 05 '16 22:11 GeThi255