Akhilesh Gotmare

icon indicating a website link https://akhileshgotmare.github.io/ icon indicating an email [email protected]

icon company Salesforce Research icon location Singapore icon location I'm an ML researcher at Salesforce Research in Singapore. Prev: ML MSc at EPFL.

Results 3 issues of Akhilesh Gotmare

masked_lm_labels -100 and not -1

Huggingface's BertForMaskedLM requires -100 for masked tokens (where no loss is to be computed/backproped) - https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L881 if set to -1, loss.backward() returns the error described here - https://github.com/pytorch/pytorch/issues/1204 and log...

Removing import WarmupLinearSchedule

Not supported by latest transformers library. See issue - https://github.com/huggingface/transformers/issues/2082 If warmup is required, we can instead use `scheduler = get_linear_schedule_with_warmup(optimizer, num_warmup_steps=WARMUP_STEPS, num_training_steps = -1)` instead of `scheduler = WarmupLinearSchedule(optimizer,...

Update README.md with ethical considerations