datasketches-java array index error on ReqSketch

ReqSketch often throws error on small dataset. only thing I explicitly set is setHighRankAccuracy(true) anyone facing this issues? its occasional so hard to understand any pattern, will try to create a test.

 java.lang.NegativeArraySizeException: -328
	at org.apache.datasketches.req.ReqCompactor.toByteArray(ReqCompactor.java:295)
	at org.apache.datasketches.req.ReqSerDe.toByteArray(ReqSerDe.java:305)
	at org.apache.datasketches.req.ReqSketch.toByteArray(ReqSketch.java:418)

Feb 05 '24 03:02 jhsenjaliya

Do you have an example of a dataset that triggers this behavior for you that we can use for validation?

Feb 05 '24 20:02 will-lauer

unfortunately I was not able to reproduce with the same dataset on single process, this is while serialization, so let me gather more info with some debug logs. Thanks for chiming in though

Feb 06 '24 01:02 jhsenjaliya

closing the issue for now. Looks like its related to the framework that is using this class whiile serialization.

Feb 19 '24 23:02 jhsenjaliya

This seems to be random actually, I thought this is due to multi threaded distributed environment where serialization is performed, but I am seeing this even in a single thread and single node. and can not figure out why the negative index would ever happen. The error comes from "ReqCompactor.java:295" on ( version 5.0.2 ) where bytes are negative that are calculated via function getSerializationBytes() -- still trying to figure out how to debug this. would really appreciate any help. Thanks

May 10 '24 18:05 jhsenjaliya

Please help us reproduce this by providing a simple example with data. --Thanks

May 22 '24 16:05 leerho

No reproducible example has been provided. We don't have evidence that this is a bug in REQ sketch. If a reproducible example can be provided, we can reopen this issue.

Jan 23 '25 01:01 leerho