NVIDIA
pulled cudaQ docker but porblem with cuda.
Required prerequisites
- [X] Consult the security policy. If reporting a security vulnerability, do not report the bug using this form. Use the process described in the policy to report the issue.
- [X] Make sure you've read the documentation. Your issue may be addressed there.
- [X] Search the issue tracker to verify that this hasn't already been reported. +1 or comment there if it has.
- [ ] If possible, make a PR with a failing test to give us a starting point to work on!
Describe the bug
I'm using a GraceHopper with aarch64. I pulled the docker: docker pull nvcr.io/nvidia/nightly/cuda-quantum:latest Then ran the container: docker run --gpus all -it nvcr.io/nvidia/nightly/cuda-quantum:latest I tried to run one one of the notebooks (deutschs_algorithm.ipynb) and got the error: ImportError: [custatevec] %the operation cannot be performed in the present state in CuStateVecCircuitSimulator (line 334)
while trying to "import cudaq"
Steps to reproduce the bug
I pulled the docker: docker pull nvcr.io/nvidia/nightly/cuda-quantum:latest Then ran the container: docker run --gpus all -it nvcr.io/nvidia/nightly/cuda-quantum:latest I tried to run one one of the notebooks (deutschs_algorithm.ipynb) and got the error: ImportError: [custatevec] %the operation cannot be performed in the present state in CuStateVecCircuitSimulator (line 334)
while trying to "import cudaq"
Expected behavior
I expected the code to run smoothly.
Is this a regression? If it is, put the last known working version (or commit) here.
Not a regression
Environment
- CUDA Quantum version:
- Python version:
- C++ compiler:
- Operating system:
Suggestions
No response
Hi @giladqm - can you please run the nvidia-smi command from within the Docker container and paste the results here?
Thanks for getting back to me so quickly: [cid:b069bcad-ff85-45cb-a247-ddc567984eb4]
From: Ben Howe @.> Sent: Wednesday, May 15, 2024 3:37 PM To: NVIDIA/cuda-quantum @.> Cc: Gilad Sivan @.>; Mention @.> Subject: Re: [NVIDIA/cuda-quantum] pulled cudaQ docker but porblem with cuda. (Issue #1681)
Hi @giladqmhttps://github.com/giladqm - can you please run the nvidia-smi command from within the Docker container and paste the results here?
— Reply to this email directly, view it on GitHubhttps://github.com/NVIDIA/cuda-quantum/issues/1681#issuecomment-2112409747, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AYGNQZ5SHP34LKWRSU6ADXTZCNJGZAVCNFSM6AAAAABHYBRLISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJSGQYDSNZUG4. You are receiving this because you were mentioned.Message ID: @.***>
That image did not come through. You may have to post it directly on the GitHub website rather than via email.
I verified that I can run the latest container on our GH200. (One possible difference is that we don't have MIG enabled, and it looks like maybe you do?) In any case, would it be possible to test with another NVIDIA container from https://catalog.ngc.nvidia.com/orgs/nvidia/teams/k8s/containers/cuda-sample? It should run like this:
$ docker pull nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda11.7.1
$ docker run --rm --gpus all nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda11.7.1
[Vector addition of 50000 elements]
Copy input data from the host memory to the CUDA device
CUDA kernel launch with 196 blocks of 256 threads
Copy output data from the CUDA device to the host memory
Test PASSED
Done
I'll check right away
From: Ben Howe @.> Sent: Wednesday, May 15, 2024 4:38 PM To: NVIDIA/cuda-quantum @.> Cc: Gilad Sivan @.>; Mention @.> Subject: Re: [NVIDIA/cuda-quantum] pulled cudaQ docker but porblem with cuda. (Issue #1681)
I verified that I can run the latest container on our GH200. (One possible difference is that we don't have MIG enabled, and it looks like maybe you do?) In any case, would it be possible to test with another NVIDIA container from https://catalog.ngc.nvidia.com/orgs/nvidia/teams/k8s/containers/cuda-sample? It should run like this:
$ docker pull nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda11.7.1 $ docker run --rm --gpus all nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda11.7.1 [Vector addition of 50000 elements] Copy input data from the host memory to the CUDA device CUDA kernel launch with 196 blocks of 256 threads Copy output data from the CUDA device to the host memory Test PASSED Done
— Reply to this email directly, view it on GitHubhttps://github.com/NVIDIA/cuda-quantum/issues/1681#issuecomment-2112564378, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AYGNQZ7AZEO3NRQOZ3VF5MDZCNQOTAVCNFSM6AAAAABHYBRLISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJSGU3DIMZXHA. You are receiving this because you were mentioned.Message ID: @.***>
It looks like this issue is not a CUDA-Q issue, but an issue with using CUDA. Can you confirm which driver version you have installed on the host (outside the container)?
I tried it again and it works: `(base) gilad@gracehopper:~$ docker run --gpus all -it nvcr.io/nvidia/quantum/cuda-quantum:0.7.1
NVIDIA CUDA-Q
=========================
Version: 0.7.1`