sparseml
sparseml copied to clipboard
Implement multiple-choice pipeline
This PR implements multiple-choice pipeline in HF's transformers library with SparseML integration. It enables commonsense-reasoning experiments proposed in the "Sparsity May Cry benchmark".
Supported datasets/tasks
- SWAG: https://arxiv.org/abs/1808.05326
- Commonsense_QA: https://arxiv.org/abs/1811.00937
- RACE: https://arxiv.org/abs/1704.04683
- Winogrande: https://arxiv.org/abs/1907.10641
Data preprocessing pipeline for each dataset follows the proposed format from the corresponding paper.
Verification
The implementation is tested by reproducing results from the RoBERTa paper (https://arxiv.org/abs/1907.11692) and fairseq repository (https://github.com/facebookresearch/fairseq/tree/main/examples/roberta).