David Papp

icon location Boston

Results 8 comments of David Papp

Inconsistent Model deploy & run performance

[INFO ] - Number of inter-op threads is 8 [INFO ] - Number of intra-op threads is 16 [INFO ] - Load PyTorch (1.9.1) in 0.003 ms. [INFO ] -...

Inconsistent Model deploy & run performance

Thanks for the help!

Inconsistent Model deploy & run performance

I ended up installing updated media codecs from RPM Fusion (including x264) on my Desktop running Fedora and things are finally working at the speeds I was expecting: [INFO ]...

Inconsistent Model deploy & run performance

Actually, this is even more interesting than I had thought. Despite the djlbench profile completing 10k iterations in 50 seconds (so 5ms), I'm finding that object detection still takes 800ms...

Exception when running model on GPU: Exception in runBots:ai.djl.engine.EngineException: Could not run 'aten::empty_strided' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build).

I forgot to mention that I'm building this project via Maven. My Maven pom.xml contains the following DJL dependencies: ``` ai.djl bom 0.14.0 pom import ... ai.djl.pytorch pytorch-engine ai.djl.pytorch pytorch-native-auto...

Exception when running model on GPU: Exception in runBots:ai.djl.engine.EngineException: Could not run 'aten::empty_strided' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build).

One more update: I tried to reinstall CUDA, and on doing so, ./gradlew debug now doesn't show CUDA as installed: ``` [DEBUG] - cudart library not found. GPU Count: 0...

Exception when running model on GPU: Exception in runBots:ai.djl.engine.EngineException: Could not run 'aten::empty_strided' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build).

Hey @frankfliu, I appreciate the suggestions. I was able to make some progress. I reinstalled CUDA and no longer have issues with DJL detecting the GPU. I realized my Maven...

Exception when running model on GPU: Exception in runBots:ai.djl.engine.EngineException: Could not run 'aten::empty_strided' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build).

Could it be an issue that I have CUDA 11.6 installed? My understanding is that minor CUDA versions should be compatible (so 11.3 should be compatible with my system's 11.6).