stakira
Add CUDA rendering support on Linux
Adds support to render using CUDA on Linux, does not touch other platforms. CUDA rendering will not be available if CUDA 12.x and cuDNN 9.x are not found and will not be listed. Tested in regular usage to ensure stability.
Was it really necessary to add both Microsoft.ML.OnnxRuntime & Microsoft.ML.OnnxRuntime.Gpu on builds? Gpu variant already have CPU EP.
Was it really necessary to add both
Microsoft.ML.OnnxRuntime&Microsoft.ML.OnnxRuntime.Gpuon builds?Gpuvariant already have CPU EP.
It was, CPU inference would crash on any OnnxRuntime from 1.18.0 up, and modern CUDA support starts with 1.18.1
How would it work after #1774?
Seeing as that updates OnnxRuntme to a newer version thats CUDA equivalent should support modern CUDA, and assuming the CPU rendering bug that was present with versions 1.18 and up is fixed with that PR, it should be possible to simply remove most of the libary loading tricker, and instead simply always load the CUDA compatible one I can experiment with it and see how it behaves
I have pushed changes that make it compatible with the latest changes in main I also removed the no longer required library loading code and confirmed CPU rendering and CUDA rendering both work
Just a question, could this not be solved with OpenCL instead, in order to enable use for all platforms and avoid implementing vendor-locked features?
https://onnxruntime.ai/docs/execution-providers/ According to the documentation here, OpenCL is not a supported execution provider of OnnxRuntime It should be fairly easy to add AMD's and Intel's equivalent render backends in the future however for people with such GPUs
Ahh I see, a bit disappointing. Checked around and see there is an effort to add one though, so that might be relevant for the future
Even if OpenCL gets support, it may still make sense to support at least some major vendor-specific backends due to increased speed over OpenCL
Is this still being worked on? I tried testing this PR on my dev build, but the Preferences window crashes with these implementations. I would love to see GPU support on Linux.
It is still being worked on as needed yes I cannot reproduce the crash, can you give me more information on the conditions for it, such as CUDA and cuDNN versions, and other specifics about the environment that may matter, as well as any other useful information that may be logged somewhere?