DeepStack icon indicating copy to clipboard operation
DeepStack copied to clipboard

Published 20 hours ago • verified publisher icon johnolafenwa

Nvidia Ampere RTX A5000 support still has isues in 2022.01.1

Open babbahotep opened this issue 3 years ago • 13 comments

Any chance to get an Ampere RTX A5000 to work? Someone posted the fix.. https://ipcamtalk.com/threads/deepstack-gpu-for-windows-problems-cuda-version-11-and-cudnn-install.59250/post-600376

I got it to work, it's a bit finicky and to do it 2 times before it worked...

babbahotep avatar Oct 29 '21 20:10 babbahotep

I am also looking for Nvidia Ampere series support. image

MikeLud avatar Nov 10 '21 19:11 MikeLud

Any chance to get an Ampere RTX A5000 to work? Someone posted the fix.. https://ipcamtalk.com/threads/deepstack-gpu-for-windows-problems-cuda-version-11-and-cudnn-install.59250/post-600376

I got it to work, it's a bit finicky and to do it 2 times before it worked...

That was my post, it also works with just doing the below steps.

Install CUDA Toolkit 11.4 cuDNN v8.2.4 DeepStack-Installer-GPU-2021.09.1.exe

Manually update to the below Windows Packages torch-1.9.1+cu111-cp37-cp37m-win_amd64 torchvision-0.10.1+cu111-cp37-cp37m-win_amd64

MikeLud avatar Nov 16 '21 03:11 MikeLud

My manual patch install keeps breaking. any chance to add the updted libraries to the release so we don't have to path this manually?

babbahotep avatar Jan 02 '22 17:01 babbahotep

Hello @babbahotep and @MikeLud , thanks for raising this. The issue with the never nvidia gpus has been fixed in this pr https://github.com/johnolafenwa/DeepStack/pull/136 . We shall be making a release before the middle of January to include this and a number of other improvements. Thanks for your patience.

johnolafenwa avatar Jan 03 '22 00:01 johnolafenwa

Awesome thank you! You guys rock!

babbahotep avatar Jan 05 '22 02:01 babbahotep

@johnolafenwa Hi John, any chance to do a quick release in Jan to roll out Ampere support?

babbahotep avatar Jan 16 '22 03:01 babbahotep

@babbahotep @johnolafenwa did say in his response "We shall be making a release before the middle of January to include this and a number of other improvements." I think any day there will be a release. image

MikeLud avatar Jan 16 '22 04:01 MikeLud

I see 2022.01.1 is out. 😄

the install documentation still says install cuda 10.1. Is that correct or should that be changed to cuda 11.6 and cudnn 8.3.2?

babbahotep avatar Jan 22 '22 04:01 babbahotep

I see 2022.01.1 is out. 😄

the install documentation still says install cuda 10.1. Is that correct or should that be changed to cuda 11.6 and cudnn 8.3.2?

I am using 2022.01.1 and I did change to cuda 11.6 and cudnn 8.3.2.

MikeLud avatar Jan 23 '22 01:01 MikeLud

There seems to be a bug in 2022.01.1 for at least RTX A5000 Ampere is not working properly yet. It works for 10-15 minutes then crashes without any logs. My setup: ESXI 7.0, VPGU, CUDA 11.6, CUDNN 8.3.2. Tested 2022.01.1 with Docker and Windows GPU. I split the GPU in half, 50% Blue IRIS Vm and 50% Deepstack ubuntu docker VM.

It just switches from 200 to 500 ERROR in the logs.

I tested on a different PC Windows GPU not virtualized with a RTX 2080. Works fine.

babbahotep avatar Feb 14 '22 16:02 babbahotep

Seems to be an issue with the vgpu stack on esxi.

Installed win 10 bare metal with regular rtx a5000 drivers, cuda and cudnn and it seems to work fine..

Any chance to try to fix this vpu driver bug? It’s probably a small user base but Id be eager to help.

babbahotep avatar Feb 15 '22 04:02 babbahotep