gpt-2-output-dataset icon indicating copy to clipboard operation
gpt-2-output-dataset copied to clipboard

Loss, Logits error while training

Open niranjanakella opened this issue 2 years ago • 1 comments

There is a assignment error in the train.py script where in the loss and logits are considered to be 'str' type after the assignment and hence have to be updated.

Line: 108 and 146

loss, logits = model(texts, attention_mask=masks, labels=labels)

Here the loss variable is assigned as a 'str' type hence the following loss.backward() would fail stating that a 'str' type doesn't have a backward method.

niranjanakella avatar Jul 13 '23 06:07 niranjanakella

This can be corrected by re-assigning the loss and logits to the corresponding model output values.

Before

loss, logits = model(texts, attention_mask=masks, labels=labels)

After

model_out = model(texts, attention_mask=masks, labels=labels)
loss, logits = model_out.loss, model_out.logits

NOTE: PR #54 has been raised

niranjanakella avatar Jul 14 '23 12:07 niranjanakella