comdb2 icon indicating copy to clipboard operation
comdb2 copied to clipboard

Published 20 hours ago • verified publisher icon bloomberg

Change nodeix-indexed arrays into hash-tables

Open markhannum opened this issue 1 year ago • 5 comments

Change arrays which are sized by MAXNODES to hash-tables so that the nodeix value isn't forced to be less than MAXNODES. This is a slightly revised take on https://github.com/bloomberg/comdb2/pull/3764 - the main difference is that it macro-izes the retrieve functions.

markhannum avatar Apr 06 '23 15:04 markhannum

/runtests

markhannum avatar Jun 14 '23 14:06 markhannum

/plugin-branch tohash_plugins

markhannum avatar Jun 14 '23 19:06 markhannum

This degrades performance significantly- I don't think we can merge this

markhannum avatar Jun 20 '23 17:06 markhannum

The transchunk_manyins test takes double the amount of time under this branch. During the course of this test, each replicant accesses the seqnum-hash 1.2 million times, while the leader accesses the coherent-state hash more than 400k times. Next highest are waitlist_counter and the udp-counters on the leader, all over 100k accesses.

I will continue to brainstorm with the team, but I don't think we can merge this as it is now.

markhannum avatar Jun 20 '23 18:06 markhannum

Keeping lockless thread-local cache in front of each hash reduces the performance hit to about 7%

markhannum avatar Jun 26 '23 15:06 markhannum