pytorch-openai-transformer-lm icon indicating copy to clipboard operation
pytorch-openai-transformer-lm copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

Confused by multiply n to both clf_logits and clf_losses?

Open Vimos opened this issue 6 years ago • 3 comments

I am confused by the code below.

https://github.com/huggingface/pytorch-openai-transformer-lm/blob/eafc28abdfadfa0732f03a0fc65805c5bfb2ffe7/train.py#L52

https://github.com/huggingface/pytorch-openai-transformer-lm/blob/eafc28abdfadfa0732f03a0fc65805c5bfb2ffe7/train.py#L54

Is this due to any normalization? Thanks!

Vimos avatar Sep 04 '18 08:09 Vimos

Yes, I don't understand that myself either. It was in the original TensorFlow code so I kept it as is. I have removed it in a fork of this code that I use for another project and it doesn't really change the results. I may try to dig further one day.

thomwolf avatar Sep 04 '18 09:09 thomwolf

Hi Thomas,

Do you have any insights now for this issue.

pranoy-k avatar Mar 11 '19 20:03 pranoy-k

I don't

thomwolf avatar Mar 11 '19 20:03 thomwolf