Faiss assertion 'err == CUBLAS_STATUS_SUCCESS' failed in void faiss::gpu::runMatrixMult(faiss::gpu::Tensor<float, 2, true>&, bool, faiss::gpu::Tensor<T, 2, true>&, bool, faiss::gpu::Tensor<IndexType, 2, true>&, bool, float, float, cublasHandle_t, cudaStream_t) [with AT = float; BT = float; cublasHandle_t = cublasContext*; cudaStream_t = CUstream_st*] at ./faiss/gpu/utils/MatrixMult-inl.cuh:133; details: cublas failed (13): (1024, 3) x (3000, 3)' = (1024, 3000)

[maxqiqq]

I use the Jupyter demo to run this project. But i encoutered this error mang times which I couldn't fix it. I changed the model and file paths. It happened when I ran the training data cell. Everytime when I ran this part, Jupyter kernel broke and restart, and the backend log showed the error:

from custom_types import CUDA, CPU from process_data.ground_truth_optimization import GroundTruthGenerator import options

args = options.GtOptions(tag='demo', mesh_name='covid', template_name='ball', num_levels=6) device = CUDA(0) gt_gen = GroundTruthGenerator(args, device) gt_gen.generate_ground_truth_meshes()

I built the environment by the yml file. Env: WSL2-Ubuntu 18.04.5 CUDA:11.8 Is there any hint??