oneDAL icon indicating copy to clipboard operation
oneDAL copied to clipboard

getBlockOfRows can not return block of rows exceeding 2 gigabytes

Open xwu99 opened this issue 3 years ago • 1 comments

We used oneDAL API for NumericTable backed by ByteBuffer, output the following message:

java.lang.IllegalArgumentException: size of the block of rows cannot exceed 2 gigabytes at com.intel.daal.data_management.data.HomogenNumericTableByteBufferImpl.getBlockOfRows(Unknown Source) at com.intel.daal.data_management.data.HomogenNumericTable.getBlockOfRows(Unknown Source)\

I checked the code:

private static final long maxBufferSize = 2147483647; long bufferSize = vectorNum * nColumns; // Gets data from C++ NumericTable object if (bufferSize * 4> maxBufferSize) { throw new IllegalArgumentException("size of the block of rows cannot exceed 2 gigabytes"); }

Is it possible to make the maxBufferSize configurable?

xwu99 avatar Mar 05 '21 03:03 xwu99

@PIVOVAR3AL

xwu99 avatar Mar 05 '21 05:03 xwu99