RuntimeError: CUDA driver error: invalid resource handle

[NickWWWW1]

When I am trying to run bash dev_scripts/dtu_test_inf/inftest_scan1.sh , i got this training images = 16 dtu_ft train id [25, 21, 33, 22, 14, 15, 26, 30, 31, 35, 34, 43, 46, 29, 16, 36] dtu_ft test id [32, 24, 23, 44] self.plane_ind 0 /home/won147/miniconda3/envs/pointnerf/lib/python3.8/site-packages/numpy/core/shape_base.py:420: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray. arrays = [asanyarray(arr) for arr in arrays] dataset total: test 4 dataset [DtuDataset] was created Traceback (most recent call last): File "train_ft.py", line 1081, in main() File "train_ft.py", line 791, in main bg_ray_train_lst = create_all_bg(train_dataset, model, img_lst, c2ws_lst, w2cs_lst, intrinsics_all, HDWD_lst) File "train_ft.py", line 572, in create_all_bg bg_ray, _ = model.set_bg(xyz_world_sect_plane, img_lst, c2ws_lst, w2cs_lst, intrinsics_all, HDWD_lst, data["plane_color"]) File "/home/won147/Desktop/pointnerf/run/../models/mvs_points_volumetric_model.py", line 284, in set_bg src_grid, mask, hard_id_xy = warp(c2w, w2c, intrinsics, xyz_world_sect_plane, HD, WD, filter=False, tolerate=0.1) File "/home/won147/Desktop/pointnerf/run/../models/mvs/mvs_utils.py", line 308, in homo_warp_nongrid mask = torch.prod(torch.cat([torch.ge(src_grid, torch.zeros([1,1,2], device=src_grid.device)), torch.le(src_grid, torch.tensor([[[WD-1,HD-1]]], device=src_grid.device))],dim=-1), dim=-1, keepdim=True, dtype=torch.int8) > 0 RuntimeError: CUDA driver error: invalid resource handle

any suggestion is appreciated!