a-PyTorch-Tutorial-to-Image-Captioning icon indicating copy to clipboard operation
a-PyTorch-Tutorial-to-Image-Captioning copied to clipboard

Published 20 hours ago • verified publisher icon sgrvinod

GPU issues

Open fly-dragon211 opened this issue 5 years ago • 0 comments

When I use muti-gpu in training, there is an issue: Some of weight/gradient/input tensors are located on different GPUs

When I revise the following code, the question is resolved! I add '.to(device)'.

       # Remove timesteps that we didn't decode at, or are pads
        # pack_padded_sequence is an easy trick to do this
        scores, _ = pack_padded_sequence(scores, decode_lengths, batch_first=True)**.to(device)**
        targets, _ = pack_padded_sequence(targets, decode_lengths, batch_first=True)**.to(device)**

        # Calculate loss
        loss = criterion(scores, targets)

        # Add doubly stochastic attention regularization
        loss += alpha_c * ((1. - alphas.sum(dim=1)) ** 2).mean()**.to(device)**

Does anyone know what happened? Is this a bug?

fly-dragon211 avatar Sep 14 '20 08:09 fly-dragon211