httpcache icon indicating copy to clipboard operation
httpcache copied to clipboard

Use fnv64a instead of sha256 for cache keys

Open hectorj opened this issue 9 years ago • 8 comments

I've done some benchmarks, and FNV is ~3 times faster than SHA256

https://gist.github.com/hectorj/8ac959c071d44ec4d718

I know it's only ~1 micro second by hashing, but it's also only a 3 lines change.

hectorj avatar Jun 14 '15 12:06 hectorj

Any idea what the impact would be in terms of cryptographic properties? Further down the track I'd like to implement cache peering, which would expose the cache keys to peers that requested them.

lox avatar Jun 28 '15 05:06 lox

Also, mind rebasing master for the test fixes?

lox avatar Jun 28 '15 05:06 lox

I'd be interested to see how MD5 compares, it's what Squid uses for cache peering: http://wiki.squid-cache.org/SquidFaq/CacheDigests

lox avatar Jun 28 '15 05:06 lox

I updated https://gist.github.com/hectorj/8ac959c071d44ec4d718 with MD5 & SHA1, and also added memory information to the results.

MD5 performs better than the SHA family, but is also known as being cryptographically insecure : https://en.wikipedia.org/wiki/MD5#Security

So if we need security I'd go with one of the SHA, and if we don't (and I don't think we do), FNV hands down

cache peering, which would expose the cache keys to peers that requested them

First time I hear about cache peering so I'm not sure, but peers are supposed to be trusted, no ?

hectorj avatar Jun 28 '15 09:06 hectorj

Rebased

hectorj avatar Jun 28 '15 09:06 hectorj

What about SipHash?

https://en.wikipedia.org/wiki/SipHash

PSUdaemon avatar Aug 05 '15 20:08 PSUdaemon

I'm fine to merge this, any objections anyone?

lox avatar Feb 23 '16 21:02 lox

I'm doing some planning on a v2.0.0, I think this would be a good point to change the hashing algorithm on keys, as I plan to change the architecture of the backends.

lox avatar Feb 23 '16 22:02 lox