CUBLAS_STATUS_NOT_INITIALIZED

[isabellapu]

Wondering what this issue means/if it is a known issue:

File "/u/ipu/miniconda3/envs/styledrop2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) File "/u/ipu/jiboji-styledrop/StyleDrop-PyTorch/open_clip/transformer.py", line 332, in forward x = r(x, attn_mask=attn_mask) File "/u/ipu/miniconda3/envs/styledrop2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/u/ipu/miniconda3/envs/styledrop2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) File "/u/ipu/jiboji-styledrop/StyleDrop-PyTorch/open_clip/transformer.py", line 252, in forward x = x + self.ls_1(self.attention(self.ln_1(x), attn_mask=attn_mask)) File "/u/ipu/jiboji-styledrop/StyleDrop-PyTorch/open_clip/transformer.py", line 247, in attention return self.attn(x, attn_mask=attn_mask) File "/u/ipu/miniconda3/envs/styledrop2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/u/ipu/miniconda3/envs/styledrop2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) File "/u/ipu/jiboji-styledrop/StyleDrop-PyTorch/open_clip/transformer.py", line 140, in forward q, k, v = F.linear(x, self.in_proj_weight, self.in_proj_bias).chunk(3, dim=-1) RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling cublasCreate(handle)