machinelearning icon indicating copy to clipboard operation
machinelearning copied to clipboard

cloning is incorrectly implemented

Open jmkinzer opened this issue 11 months ago • 1 comments

The CloneAs* methods for PrimitiveDataFrameColumn<T> are incorrect. They seem to assume 1 input buffer will become 1 output buffer. This can't be correct in the case where for example:

  • The original buffer is at MaxCapacity elements => therefore has size in bytes of ArrayUtility.ArrayMaxSize
  • The new buffer has a larger sizeof(T)

As an experiment I set the ArrayMaxSize to a smaller value (say 32768). In integer elements this will therefore be 8192 but in double elements only 4096. However the clone routine will still attempt to allocate a buffer of 8192 doubles = 65K bytes. Running an operation that would cause a clone I indeed see the expected exception:

Image

jmkinzer avatar Feb 06 '25 19:02 jmkinzer

@luisquintanilla @JakeRadMSFT thoughts on this? I'll tag this for the Future milestone since I don't think I can get to it for this release, unless someone else wants to pick it up.

michaelgsharp avatar Mar 19 '25 16:03 michaelgsharp