oneDAL icon indicating copy to clipboard operation
oneDAL copied to clipboard

getBlockOfRows can not return block of rows exceeding 2 gigabytes

Open xwu-intel opened this issue 4 years ago • 1 comments

We used oneDAL API for NumericTable backed by ByteBuffer, output the following message:

java.lang.IllegalArgumentException: size of the block of rows cannot exceed 2 gigabytes at com.intel.daal.data_management.data.HomogenNumericTableByteBufferImpl.getBlockOfRows(Unknown Source) at com.intel.daal.data_management.data.HomogenNumericTable.getBlockOfRows(Unknown Source)\

I checked the code:

private static final long maxBufferSize = 2147483647; long bufferSize = vectorNum * nColumns; // Gets data from C++ NumericTable object if (bufferSize * 4> maxBufferSize) { throw new IllegalArgumentException("size of the block of rows cannot exceed 2 gigabytes"); }

Is it possible to make the maxBufferSize configurable?

xwu-intel avatar Mar 05 '21 03:03 xwu-intel

@PIVOVAR3AL

xwu-intel avatar Mar 05 '21 05:03 xwu-intel