Add LRA to CI

[dianaml0]

🚀 Feature

Capturing idea from @blefaudeux here. Add LRA to the CI since some bugs are only showing up through manual runs of it. Should be a smaller version of the full LRA.

Motivation

LRA has caught many bugs, could be useful to add it to the continuous integration testing.

Pitch

LRA with a smaller dataset.

Alternatives

Limit tasks and attention variants to a subset. Run a benchmark in parallel not necessarily linked to CI

Additional context

https://github.com/facebookresearch/xformers/pull/145#issuecomment-996223858

[blefaudeux]

I think that one plan could be to:

• add minGPT like (pull shakespeare, autoregressive task) as an LRA task (same formalism as the others)
• use that as a lightweight task on CI, over a couple of tokens, checking that some of the LRA codepath is in good shape

thoughts ?

[dianaml0]

Yes that sounds like a good idea! Should we also have the full LRA run regularly as well not necessarily part of CI?

[blefaudeux]

Yes agreed for the full LRA, but then probably on the Fair cluster with the framework built by Caleb Ho ?