NDArray creation bug

[onslauth]

The following doesn't work:

  var n = new NDArray( b, new Shape( 64, 32, 32, 3 ), TF_DataType.TF_FLOAT ) // b is byte array of size 196608

Error: Exception thrown at 0x00007FF8C0FE7E20 (tensorflow.dll) in ScanIT.exe: 0xC0000005: Access violation reading location 0x0000000000000000.

The following works:

var n = new NDArray( b )
n = n.reshape( new Shape( 64, 32, 32, 3 ) );
n = n.astype( TF_DataType.TF_FLOAT );

[m0nzderr]

Actually, this looks almost fine to me. In your first example you are trying to create an array of 196608 floats, but provide only 196608 bytes (4 times less than necessary). Perhaps, in this case, byte[] is interpreted as the data buffer, not the values to be converted later to floats. So it it ok to have an exception here.

The not-so-good part is that the kind of an exception thrown by memory manager (unrecoverable crash). Another problem is the exact behavior is not clear, and there is not much said about memory management in the documentation. For those who came from python the confusion is even bigger, when one finds out that NDarray is being inherited from a Tensor.

What should be done about it? My suggestion is to start from making it clear how things work: NDArray(byte[] bytes, Shape shape, TF_DataType dtype)

Are bytes[] a binary buffer data or values to be converted? Who's responsibility is to perform basic boundary checks? I'd expect some basic checks should be done in the constructor (i.e., assert bytes.length == numel(shape) * dtype.size) before allocating the unmanaged memory.

[Oceania2018]

@onslauth Please reopen this issue if you still have any problem.