KernelAbstractions.jl icon indicating copy to clipboard operation
KernelAbstractions.jl copied to clipboard

Add backend lookup function based on input arguments

Open jpsamaroo opened this issue 4 years ago • 2 comments

To eliminate code like https://github.com/SciML/DiffEqGPU.jl/blob/6c1886bf4e62a6293490d6c333a1302045a744d0/src/DiffEqGPU.jl#L369-L370, it would be nice if we could select the backend for the user automatically based on the input arguments provided. I propose an Auto() backend which can do this selection. With the fallback being CPU(), we'd select an appropriate GPU backend if any arguments contain things like CuArray/ROCArray, but would throw an error if both kinds are mixed (unlikely, but who knows, maybe someone has an NVidia+AMD machine).

jpsamaroo avatar Mar 30 '21 14:03 jpsamaroo

I have punted on this in the past because:

  1. Launch configuration is device specific
  2. Array allocations come from the user
  3. Arbitrary nested structs

vchuravy avatar Mar 30 '21 14:03 vchuravy

Good points!

  1. I wonder if we should also try to "solve" this by determining a good launch configuration based on occupancy calculations (which could be requested from the GPU backends by way of specifying a keyword like :auto.
  2. Some libraries (like DiffEqGPU) don't expose backend selection to the user, so the user must generally rely on dispatch within the library to select the right backend. I would argue that Auto() makes us more idiomatic for such cases.
  3. Yeah, I wasn't sure what to do about this. I feel like we could address this by first using Adapt to recursively check for matching types (and possibly do the replacement at that point), which could compile out for type-stable arguments.

I know that KA doesn't want to do too much that really should be left up to the user, but I feel like if we want to be a very thin abstraction layer, we should provide a way to "hide" things like backend and workgroupsize for users who want a "seamless" experience.

jpsamaroo avatar Mar 30 '21 15:03 jpsamaroo

I think this is fixed with #320 ?

vchuravy avatar Sep 23 '22 09:09 vchuravy