Randomly getting error while doing t.tensor(gt_roi_label).detach().cuda()

[IISCAditayTripathi]

trafficstars

I am getting the following error while pushing the variable gt_roi_label to cuda. Sometimes this works and sometimes this doesn't throw error.

when I do t.tensor(gt_roi_label), it always works but when I do t.tensor(gt_roi_label).detach().cuda() or t.tensor(gt_roi_label).cuda() it works sometimes and other times it gives following error

  File "train_sketch.py", line 81, in train
trainer.train_step(img, bbox, label, scale, sketches)
File "/scratche/home/aditay/visual_grounding_work/finding_beans_code/dsve-loc/new_faster_rcnn/simple-faster-rcnn-pytorch/trainer_sketch.py", line 176, in train_step
File "/scratche/home/aditay/visual_grounding_work/finding_beans_code/dsve-loc/new_faster_rcnn/simple-faster-rcnn-pytorch/trainer_sketch.py", line 176, in train_step
losses = self.forward(imgs, bboxes, labels, scale, sketches)
File "/scratche/home/aditay/visual_grounding_work/finding_beans_code/dsve-loc/new_faster_rcnn/simple-faster-rcnn-pytorch/trainer_sketch.py", line 152, in forward
gt_roi_label = t.tensor(gt_roi_label).detach().cuda()
RuntimeError: CUDA error: device-side assert triggered