Issue with formaula of BCE loss in 06_multicat.ipynb

[ggsharma]

trafficstars

Hey,

I was just going through the book and found a functions as

def binary_cross_entropy(inputs, targets):
    inputs = inputs.sigmoid()
    return -torch.where(targets==1, 1-inputs, inputs).log().mean()

According to me the function should return -torch.where(targets==1, inputs, 1-inputs).log().mean() since the BCE loss is defined as (https://pytorch.org/docs/stable/generated/torch.nn.BCEWithLogitsLoss.html)

# Example
train_y = torch.tensor([[1],[1],[1],[1],[0],[0]],dtype=torch.float32)

preds = torch.tensor([[0.5],
        [0.4],
        [0.7],
        [1.5],
        [2.4],
        [-2.7]],dtype=torch.float32)

criterion = torch.nn.BCEWithLogitsLoss()

inputs = preds.sigmoid()

ans1 = -torch.where(train_y == 1, 1-inputs, inputs).log().mean()
ans2 = criterion(preds, train_y)
ans3  = -torch.where(train_y == 1, 1, 1-inputs).log().mean()

print(ans1) # gives 1.2573 as ans
print(ans2) # gives 0.6906 as ans
print(ans3) # gives 0.6906 as ans

[yao-eastside]

Thanks for reporting the issue. I have created a PR here: https://github.com/fastai/fastbook/pull/480

BTW, There is a typo in your post: ans3 = -torch.where(train_y == 1, 1, 1-inputs).log().mean() should be ans3 = -torch.where(train_y == 1, inputs, 1-inputs).log().mean()