MXNet.jl icon indicating copy to clipboard operation
MXNet.jl copied to clipboard

No tools for many index operations DArrays

Open han-so1omon opened this issue 6 years ago • 1 comments

From the http://dmlc.ml/MXNet.jl documentation as well as reading source code, the tools for comparison-based index operations on NDArrays are not apparent. Because NDArray is not compatible with common Julia index methods find(), etc the alternative is to write a method. However, there is no way to retrieve a conditional out of an NDArray, even an array of ones/zeros.

For example how would one find the index of the maximum element of an array? The mx.maximum() function returns the maximum element, and the mx._equal() function can do NDArray element-wise comparison. Thus, an NDArray of ones or a single element NDArray is found, but this can't be used as a Julia-compatible Bool.

This seems like a major flaw, and so it must be the case that I'm missing something simple here.

Edit: argmax(), etc are in dlmc.ml documentation, but they throw errors:

Check failed: mshadow::DataType<DType>::kFlag == type_flag_ TBlob.get_with_shape: data type do not match specified type.Expected: 6 v.s. given 0 ...

Edit2: Found the @mx.nd_as_jl macro. If this is the only way to do index operations (i.e. while argmax(), etc are being fixed/implemented), then it should be highlighted more in documentation.

han-so1omon avatar Apr 23 '18 19:04 han-so1omon

Hi @han-so1omon ,

Because NDArray is not compatible with common Julia index methods find(), etc the alternative is to write a method. However, there is no way to retrieve a conditional out of an NDArray, even an array of ones/zeros.

I think the simplest way to do that is copy the NDArray to a normal Julia Array (e.g. find(copy(arr)), it's same as your Edit2).

But if you want element-wised comparison of NDArray, it's implemented. Maybe you can fit your code into this form.

julia> x = NDArray([1,2,3,4])                                                                                                                     
4-element mx.NDArray{Int64,1} @ CPU0:                                                                                                             
 1                                                                                                                                                
 2                                                                                                                                                
 3                                                                                                                                                
 4                                                                                                                                                
                                                                                                                                                  
julia> y = NDArray([1,2,5,6])                                                                                                                     
4-element mx.NDArray{Int64,1} @ CPU0:                                                                                                             
 1                                                                                                                                                
 2                                                                                                                                                
 5                                                                                                                                                
 6                                                                                                                                                
                                                                                                                                                  
julia> x .== y                                                                                                                                    
4-element mx.NDArray{Int64,1} @ CPU0:                                                                                                             
 1                                                                                                                                                
 1                                                                                                                                                
 0                                                                                                                                                
 0

About argmax: that's indeed a bug... There are still lots of function need to be take care: https://travis-ci.org/dmlc/MXNet.jl/jobs/341761962#L1947-L1949 Because C/Cpp are 0-based index.

iblislin avatar Apr 25 '18 09:04 iblislin