pytorch-lightning-transformers Medium article

Setup

$ git clone https://github.com/sobamchan/pytorch-lightning-transformers.git
$ cd pytorch-lightning-transformers
$ pipenv install
$ pipenv shell

Usage

Fine-tune for CommonsenseQA with 2 gpus.

CUDA_VISIBLE_DEVICES=0,1 python csqa.py --gpus 2

Fine-tune for MRPC

$ python mrpc.py

This will load pre-trained BERT and fine-tune it with putting classification layer on top on MRPC task (paraphrase identification).