MLUtils.jl icon indicating copy to clipboard operation
MLUtils.jl copied to clipboard

`CuArray`s should not follow fast paths in `obsview`

Open darsnack opened this issue 3 years ago • 3 comments

CuArrays should not follow the fast path in obsview that returns a view of the original array, since this triggers scalar indexing for downstream operations.

ref https://github.com/FluxML/Flux.jl/issues/1935

darsnack avatar Apr 08 '22 13:04 darsnack

I guess the main issue here is taking on CUDA as a dep. The fix is easy.

darsnack avatar Apr 08 '22 13:04 darsnack

I'm filing a PR with the fix

CarloLucibello avatar Apr 08 '22 13:04 CarloLucibello

PR #73 fixed the DataLoader issue but the general problem with CuArrays remains. I opened https://github.com/JuliaGPU/CUDA.jl/issues/1472. If that cannot be solved as it is likely to be, I can think of the following workarounds:

Option 1. obsview never returns views for AbstractArrays

In this case, we always get an ObsView object.

PROS

  • consistent behavior for all arrays

CONS

  • cannot work with obsviews as arrays anymore

Option 2. Opt-in: obsview returns views only for selected cpu array types

PROS

  • it will still be possible to perform array operations with obsviews of a few types

CONS

  • inconsistent behavior
  • lengthy implementation, have to account for Array, SparseVector, SparseMatrixCSC, Adjoint, Transpose

Option 3. Opt-out of AbstractGPUArrays: obsview doesn't returns views for selected gpu arrays

PROS

  • it will still be possible to perform array operations with obsviews of any cpu type

CONS

  • inconsistent behavior
  • doesn't play nicely with wrappers
julia> x = CUDA.zeros(5)'
1×5 adjoint(::CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}) with eltype Float32:
 0.0  0.0  0.0  0.0  0.0

julia> x isa CUDA.AbstractGPUArray
false

CarloLucibello avatar Apr 09 '22 07:04 CarloLucibello