pytorch-transformers-classification

pytorch-transformers-classification copied to clipboard

Model performance degrades when moved to Multi-GPU

5

Hi, When I run your code on multi-gpu, performance degrades severely (compared to the single-gpu version). To make the code multi-gpu competable, I've only added 2 lines of code: *...

ereday

TypeError: convert_examples_to_features() got an unexpected keyword argument 'sep_token_extra'

6

I am getting this error when I run: `if args['do_train']: train_dataset = load_and_cache_examples(task, tokenizer) global_step, tr_loss = train(train_dataset, model, tokenizer) logger.info(" global_step = %s, average loss = %s", global_step, tr_loss`...

Mahhos

AttributeError: module 'torch.nn.functional' has no attribute 'one_hot'

1

if args['do_train']: train_dataset = load_and_cache_examples(task, tokenizer) global_step, tr_loss = train(train_dataset, model, tokenizer) logger.info(" global_step = %s, average loss = %s", global_step, tr_loss) I still get this error when running this...

nalcvp

How to run utils.py?

3

Is there any specific parameter that we should pass to run the utils.py? Running the file itself seems does nothing. It does not create the InputFeature objects or anything. What...

Mahhos

Validating the model

7

Hi, I would like to know if the model over-fits and also the optimum number of epochs, plotting accuracy and loss as it's shown [here](https://stackoverflow.com/questions/41908379/keras-plot-training-validation-and-test-set-accuracy). It would be possible to...

Magpi007

SummaryWriter Import Missing from Gist

3

Hello! The following import line is missing from the Gist you provided in your article. Discovered it was missing when I got an error running the code. I found it...

alfredtangsw

How to avoid CUDA out of memory error for large batch sizes?

1

I have two GPUs (2 x NVIDIA Tesla V100) and I'm running the codes in `run_model.ipynb` on Google Cloud. I get the CUDA out of memory exception when I want...

phosseini

Extensions

13

Thanks to your help - I have added custom losses, special initialization and a bunch of other things as extensions. I am now trying to mess with the sentence classification...

pythonometrist

AttributeError: module 'torch.nn.functional' has no attribute 'one_hot'

5

Hi, I downloaded and ran your program, and got a training error as above. I have no GPU, so I changed the setup to fp16 = 'false' (xlnet left as...

wongdarrell

Update prerec packages info: transformer and apex?

1

1. replacing every pytorch_transformers by transformers fixes the problem of giving negative dimensions to create tensor. 2. fp 16 requires nv apex, but can you write something to note that...

HaoLi111