SAM (sometimes) works on NLP too + SAM can be combined with SWA

[JeanKaddour]

Two minor suggestions for your model cards on SAM/SWA:

• We found SAM to be effective on many NLP tasks too; sometimes also on GNN tasks, see Tables 2 and 3, respectively, in this paper for more information.
• In the same paper, we also combined SAM and SWA (which seems straightforward to implement with your excellent Composer trainer) and empirically demonstrated that it improves over either approach in 39 out of 42 cases.

Thanks a lot for creating this exciting library!

[hanlint]

@JeanKaddour thanks for pointing this out, and the link to the paper. Indeed, combining SAM and SWA sounds promising, we will take a look!

[mvpatel2000]

Closing. Tracking elsewhere as low pri. We're open to community suggestions!