graphite-web icon indicating copy to clipboard operation
graphite-web copied to clipboard

Feature request: support for jump_fnv1a_ch hashing algorithm

Open piotr1212 opened this issue 8 years ago • 14 comments

The consistent hashring sharding is very unbalanced as discussed here: https://github.com/graphite-project/carbon/issues/485 @grobian implemented jump_fnv1a_ch in https://github.com/grobian/carbon-c-relay which is better balanced.

It would be nice if graphite-web would support jump_fnv1a_ch.

piotr1212 avatar Aug 11 '16 09:08 piotr1212

Please re-open this issue under the https://github.com/graphite-project/carbon project.

obfuscurity avatar Aug 11 '16 12:08 obfuscurity

My request is for graphite-web to support jump_fnv1a so I can use graphite-web in combination with carbon-c-relay in jump_fnv1a_ch mode. Now graphite-web's carbonlink queries fail because they query the wrong cache.

From what I can tell the hashring (used for carbonlink queries) is implemented here: https://github.com/graphite-project/graphite-web/blob/master/webapp/graphite/render/hashing.py

I could open a second issue on carbon to request it implemented in carbon-relay, but I personally don't need it as I don't use carbon-relay. I'm not sure if I understand why this issue is closed.

piotr1212 avatar Aug 11 '16 14:08 piotr1212

My apologies, I misunderstood your request.

obfuscurity avatar Aug 11 '16 14:08 obfuscurity

Thinking more about it, it would make sense to have the ConsistentHashRing class not duplicated in the two projects. ConsistentHashRing could be defined in carbon and imported in graphite-web (unless you want to prevent dependencies between the projects).

A quick check between https://github.com/graphite-project/graphite-web/blob/master/webapp/graphite/render/hashing.py#L45-L100 and https://github.com/graphite-project/carbon/blob/master/lib/carbon/hashing.py#L8-L62 shows that there are slight differences. I haven't looked into details so don't know if there are issues, but I would expect it to be identical.

piotr1212 avatar Aug 11 '16 15:08 piotr1212

Both carbon and graphite-web has support of user keyfunc for hashing, not sure if that enough for proper fnv1a support, suspecting that's not...

deniszh avatar Aug 11 '16 16:08 deniszh

We really could use this feature. We use the fnv1a_ch from Carbon-C-Relay to make our cluster easy to scale out, since we are not dependent on hostnames and ports.

bemeyert avatar Oct 10 '16 13:10 bemeyert

I added FNV1a support in PR https://github.com/graphite-project/graphite-web/pull/1723

deniszh avatar Oct 14 '16 15:10 deniszh

Amazing!! thanks

piotr1212 avatar Oct 15 '16 11:10 piotr1212

Fixed by #1723.

obfuscurity avatar Oct 17 '16 03:10 obfuscurity

Just a note - FNV1a != Jump FNV1a, so the issue is not fixed.

Civil avatar Nov 18 '16 00:11 Civil

Do we have any plans to support jump_fnv1a_ch hashing algorithm in graphite-web?. @deniszh

cjagus avatar Apr 02 '19 19:04 cjagus

Well, I can try. I do not remember what stopped me, probably complete different implementation of hash ring for jump hash.

deniszh avatar Apr 02 '19 19:04 deniszh

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Apr 13 '20 20:04 stale[bot]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Jun 19 '20 22:06 stale[bot]