Stephen Roller

Retire custom AdamWithDecay

1

We have a custom Adam optimizer that says it could be removed via pytorch 1.2: https://github.com/facebookresearch/ParlAI/blob/3a5de86667816abbdb0f08049c86d1e2c3aff03e/parlai/agents/bert_ranker/helpers.py#L242-L350 The task is to remove it and upgrade to the official pytorch version. The...

Immutable opt

1

Opt should be made immutable. - Remove its `dict` superclass - Similar to message, throw a hissy fit on `__setitem__` - Add a new fork option, which allows for values...

HuggingFace Transformers interop

1

HuggingFace's [transformers](https://github.com/huggingface/transformers) has a lot of great models built in. We can make a generic interop to it. I believe we want both a `huggingface/ranker` and `huggingface/generator`. We might need...

Opt should remember where options were defined

1

Might be more convenient than the usual way we grep for `--option.without.underscores` This could also be possibly used to cull opts that don't belong.

Unify names of some options

1

Now that we have `upgrade_opt` we can do some resolution of the `embedding_size` and `embeddingsize` madness we have in seq2seq vs transformer vs memnn etc.

ParlaiParser.add_argument should support no_save and no_override

1

Depends on #2172. - If no_save is True, then the option is not saved with the opt. - If no_override is True, then the parser will throw a hard error...

DSTC7 teacher should be named Ubuntu_DSTC7

1

The name is misleading and confusing. We should rename it, and expand `-t dstc7` to indicate ALL DSTC7 tasks.

**Patch Description** Since we're doing manual activation checkpointing, we need to have custom backwards for MHA. This patch leverages the flash implementation in xformers. TODO: - [ ] Gate behind...

**Patch Description** Due to changes of logic around NFS handling, config.yml is no longer being saved along with training runs. Small hack to fix that. **Testing steps** Launched a 1...

**Patch Description** Add @sharannarang to CODEOWNERS **Testing steps** n/a