sherpa-onnx
sherpa-onnx copied to clipboard
k2-fsa
I can’t run sherpa-onnx on the GPU in C#, and just like that
Good afternoon.
I'm trying to run sherpa-onnx in my C# project (using the nuget package org.k2fsa.sherpa.onnx)
And not just run it on the CPU (everything works well on it and there are no obvious problems), but run it on the GPU (I have a 1080Ti)!
For this:
-
In my program code I set the provider parameter to cuda (screenshot attached)
-
Installed CUDA 12.4 on my computer (I also tried 11.8)
-
Collected and recompiled sherpa-onnx according to your instructions (link https://k2-fsa.github.io/sherpa/onnx/install/windows.html)
-
I tried to put the resulting libraries into folders (in different places) of my project, but it still didn’t run on the GPU (I attach a screenshot of the message)
Moreover:
I even downloaded your finished project (via this link https://github.com/k2-fsa/sherpa-onnx/releases/download/v1.10.1/sherpa-onnx-v1.10.1-win-x64-cuda.tar.bz2)
I unzipped it and entered all the necessary parameters (this can be seen in the provided screenshots) and still nothing starts on the GPU and doesn’t even write any errors
At the same time, if I set the provider=cpu flag, then everything works (screenshot attached)!
What could be the problem?
For sherpa-onnx-offline.exe, does it use GPU when you use --provider=cuda?
The logs in your screenshot do not show it is using CPU.
For The C# part, could you unzip https://github.com/k2-fsa/sherpa-onnx/releases/download/v1.10.1/sherpa-onnx-v1.10.1-win-x64-cuda.tar.bz2) and copy all .dll files from it and use them to replace all the .dll files in your C# project?
I downloaded the archive from this link https://github.com/k2-fsa/sherpa-onnx/releases/download/v1.10.1/sherpa-onnx-v1.10.1-win-x64-cuda.tar.bz2
Unzipped it. I took it from it (from the bin folder) all files with *.dll extension
And put it in the root of the SherpaApp.exe file of my project Along the way C:\C#\Sherpa\SherpaApp\bin\Debug\net8.0 (screenshot attached)
In the code of my project I explicitly indicated (already indicated) provider=cuda
I tried to launch...
And nothing! Exactly the same as before, the message that
Exactly the same message as before..."bla bla bla ...Fallback to cpu!" (screenshot attached)
Then I tried to copy all *.dll to the native folder
Along the way C:\C#\Sherpa\SherpaApp\bin\Debug\net8.0\runtimes\win-x64\native (screenshot attached)
The result is exactly the same!(((
I have CUDA installed (screenshot attached)
What could be the problem?
For sherpa-onnx-offline.exe, does it use GPU when you use
--provider=cuda? The logs in your screenshot do not show it is using CPU.
I did not understand the question
What are you about?
The above screenshot shows you are using sherpa-onnx-offline.exe and you have provided --provider=cuda.
The logs do no show anything related about the CPU. So I am asking if this command is using GPU.
If not, please tell us why you think it is not using GPU.
I suggest you have a look at the existing issue that has already been resolved. https://github.com/k2-fsa/sherpa-onnx/issues/878
I have CUDA installed
That is not enough. There are various versions of CUDA. You need to install a specific version. Please see the above issue.
The above screenshot shows you are using
sherpa-onnx-offline.exeand you have provided--provider=cuda. The logs do no show anything related about the CPU. So I am asking if this command is using GPU. If not, please tell us why you think it is not using GPU.
The above screenshot shows you are using
sherpa-onnx-offline.exeand you have provided--provider=cuda. The logs do no show anything related about the CPU. So I am asking if this command is using GPU. If not, please tell us why you think it is not using GPU.
No no no
This screenshot is just an attempt to run it on the GPU (cuda), but...
She's not successful!
After the message
Creating recognizer...
Nothing happens!
And yes, this is an attempt to run on a GPU!
I suggest you have a look at the existing issue that has already been resolved. #878
I have CUDA installed
That is not enough. There are various versions of CUDA. You need to install a specific version. Please see the above issue.
Thank you, I'll take a look!
I suggest you have a look at the existing issue that has already been resolved. #878
I have CUDA installed
That is not enough. There are various versions of CUDA. You need to install a specific version. Please see the above issue.
I read and tried everything that the author of this post did!
Nothing helped me!
Doesn't run on GPU!
Moreover, like my project, so does your downloaded release sherpa-onnx-v1.10.1-win-x64-cuda
But I have a question!
Why do you have nuget package org.k2fsa.sherpa.onnx
But there is no nuget package org.k2fsa.sherpa.onnx-GPU!
I think this is the problem!
Give me the nuget package org.k2fsa.sherpa.onnx-GPU!
The above screenshot shows you are using
sherpa-onnx-offline.exeand you have provided--provider=cuda. The logs do no show anything related about the CPU. So I am asking if this command is using GPU. If not, please tell us why you think it is not using GPU.
The above screenshot shows you are using
sherpa-onnx-offline.exeand you have provided--provider=cuda. The logs do no show anything related about the CPU. So I am asking if this command is using GPU. If not, please tell us why you think it is not using GPU.No no no
This screenshot is just an attempt to run it on the GPU (cuda), but...
She's not successful!
After the message
Creating recognizer...
Nothing happens!
And yes, this is an attempt to run on a GPU!
You are the first one reporting that sherpa-onnx does not work on Windows GPU.
Please try to build sherpa-onnx from source by following our doc and see if it works for you.
sherpa-onnx-offline.exeand you have provided--provider=cuda. The logs do no show anything related about the CPU. So I am asking if this command is using GPU. If not, please tell us why you think it is not using GPU.
sherpa-onnx-offline.exeand you have provided--provider=cuda. The logs do no show anything related about the CPU. So I am asking if this command is using GPU. If not, please tell us why you think it is not using GPU.No no no This screenshot is just an attempt to run it on the GPU (cuda), but... She's not successful! After the message Creating recognizer... Nothing happens! And yes, this is an attempt to run on a GPU!
You are the first one reporting that sherpa-onnx does not work on Windows GPU.
Please try to build sherpa-onnx from source by following our doc and see if it works for you.
I made a video on youtube, where I show that even your project does not work for me! https://youtu.be/2i7O2qpLRzU?si=oaIkcTaNpBmZUJbE
Please help me!
@sanitar1988 Thank you for taking the time to make a video to describe your issue.
Could you have a look at https://github.com/k2-fsa/sherpa-onnx/issues/878?
Specifically, you are using cuda 12.4, which causes the issue that you have.
Please don't use cuda 12.4
Please have a look at
- https://github.com/k2-fsa/sherpa-onnx/issues/878#issuecomment-2111520933
- https://github.com/k2-fsa/sherpa-onnx/issues/878#issuecomment-2111549998
- https://github.com/k2-fsa/sherpa-onnx/issues/878#issuecomment-2111565192
- https://github.com/k2-fsa/sherpa-onnx/issues/878#issuecomment-2111577268
@sanitar1988 Thank you for taking the time to make a video to describe your issue.
Could you have a look at #878?
Specifically, you are using cuda 12.4, which causes the issue that you have.
Please don't use cuda 12.4
Please have a look at
So I have already tried everything that is written in the topic at the link!
Literally a little higher by 6 messages!
@sanitar1988 Thank you for taking the time to make a video to describe your issue.
Could you have a look at #878?
Specifically, you are using cuda 12.4, which causes the issue that you have.
Please don't use cuda 12.4
Please have a look at
I tried both CUDA version 11.8 and 12.4
It doesn’t work with either 11.8 or 12.4 versions!
@csukuangfj
Looking in onnxruntime/ci_build/build.py#L1173
I may wrong, but it looks like it requires onnxruntime_USE_CUDA to be ON and I don't see any place that sherpa pass it.
I'm digging there because I'm trying to figure out how to enable rocm provider there, how to pass onnxruntime_USE_ROCM=ON from sherpa cmake
Also you can see in his screenshot that only cpu provider is available
I may wrong, but it looks like it requires onnxruntime_USE_CUDA to be ON and I don't see any place that sherpa pass it.
Note that when you build onnxruntime from source, you have to pass onnxruntime_USE_CUDA=ON.
However, when you use a pre-built onnxruntime lib in sherpa-onnx, you don't need to do that. Even if you do that, it would be ignored by sherpa-onnx.
We have to find a pre-built onnxruntime lib that has rocm inside. I will try to build one using github actions. I will let you know whether I can make it or not as soon as possible.
Note that when you build onnxruntime from source, you have to pass
onnxruntime_USE_CUDA=ON.
Oh now I see why it doesn't exists, we use pre build onnxruntime, thanks.
We have to find a pre-built onnxruntime lib that has rocm inside. I will try to build one using github actions. I will let you know whether I can make it or not as soon as possible.
Maybe this will help https://github.com/pykeio/ort/blob/main/ort-sys/dist.txt Not sure if their build script are open source
I may wrong, but it looks like it requires onnxruntime_USE_CUDA to be ON and I don't see any place that sherpa pass it.
Note that when you build onnxruntime from source, you have to pass
onnxruntime_USE_CUDA=ON.However, when you use a pre-built onnxruntime lib in sherpa-onnx, you don't need to do that. Even if you do that, it would be ignored by sherpa-onnx.
We have to find a pre-built onnxruntime lib that has rocm inside. I will try to build one using github actions. I will let you know whether I can make it or not as soon as possible.
@sanitar1988 Thank you for taking the time to make a video to describe your issue.
Could you have a look at #878?
Specifically, you are using cuda 12.4, which causes the issue that you have.
Please don't use cuda 12.4
Please have a look at
Hi, I tried to compile and run everything on the GPU!
Following your advice I installed 11.8 cuda
But nothing works, anyway!
Here's a video https://youtu.be/Zjvb56UIRmo
Following your advice I installed 11.8 cuda
Could you show in the video where you installed cuda 11.8?
Also, please output English logs in the video when it is not working.
Following your advice I installed 11.8 cuda
Could you show in the video where you installed cuda 11.8?
Also, please output English logs in the video when it is not working.
The video shows everything!
In the interval from 18:00 to 18:24 minutes - I show that I have 11.8 cuda installed
What does "...output English logs in the video when it is not working." mean?
Explain in more detail