honlzl

RuntimeError: CUDA error: device-side assert triggered

1

FUNSD, lilt-roberta-en-base return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse) RuntimeError: CUDA error: device-side assert triggered 7%|▋ | 143/2000 [00:28

RuntimeError: weight tensor should be defined either for all 4 classes or no classes but got weight tensor of shape: [3]

4

When I set train_batch_size=1, I got this error. It happen in ``` n_loss = compute_crossentropy_loss(n_scores.to(device), tg.ndata['label'].to(device)) ==> def compute_crossentropy_loss(scores: torch.Tensor, labels: torch.Tensor): w = class_weight.compute_class_weight(class_weight='balanced', classes=np.unique(labels.cpu().numpy()), y=labels.cpu().numpy()) return torch.nn.CrossEntropyLoss(weight=torch.tensor(w, dtype=torch.float32).to('cuda:0'))(scores,...

The more training rounds, the lower the GPU usage rate

5

I wanted test speed in python. I tried to replicate gpu(L40) and cpu(28cores) experiment with higgs. The following are the experimental results. num_iterations(500): cuda(28s) version was slower than cpu(71s). num_iterations(5000):...