lz4-java icon indicating copy to clipboard operation
lz4-java copied to clipboard

(unnecessary?) sync locking when checking if Native is loaded causes unneeded slowdown

Open noamBarkai opened this issue 9 years ago • 5 comments
trafficstars

When using this library in a highly concurrent setting - vis. Apache Spark - I'm seeing a lot of contention around this synchronisation block, specifically when called from here It seems the synchronisation is somewhat redundant and that it can safely be removed, assuming the boolean member loaded is modified to volatile, no? I can offer a PR if you feel like wise.

noamBarkai avatar Feb 21 '16 10:02 noamBarkai

Yes, I think changing loaded to volatile will work. If you still see the contention problem and if you submit a PR, I'll take a look at it.

odaira avatar Sep 17 '18 20:09 odaira

@odaira - it's been a while since this issue was opened... :) I'm not sure if this problem still persists, but regardless I can open a PR, no problem, I don't think it's harmful.

noamBarkai avatar Sep 25 '18 09:09 noamBarkai

https://github.com/lz4/lz4-java/pull/126

noamBarkai avatar Sep 25 '18 09:09 noamBarkai

Thanks much for your contribution. Because I am about to release a new version, I'll review this after that.

odaira avatar Sep 25 '18 16:09 odaira

In Spark, https://github.com/apache/spark/pull/24905 may partially mitigate this by significantly reducing the number of calls to fastestInstance().

JoshRosen avatar Jun 23 '19 20:06 JoshRosen