electra

electra copied to clipboard

Hi, I have a new corpus and want to fine-tune electra for better result. Can run file run_mlm.py makes my pre-trained electra works better

ToanKGO

How can I draw this?

1

can you help me? how can I train the model with eval? I use tensorboard, the loss is only one。 

zshy1205

Using own data to continue pre-training from the released ELECTRA checkpoints

4

Hello, I would like to pretrain using my own corpus but starting from the released Electra model weights. In the Readme file, it says I should download the pretrained weights...

ghost

Cannot import trace from tensorflow.python.profiler

4

Just installed tensorflow 1.15 using conda, getting this error when attempting to run the pre-training command provided in the quickstart section of the readme. ``` Traceback (most recent call last):...

n-garc

Why Tagging Task segment ids are ones instead of zeros? https://github.com/google-research/electra/blob/8a46635f32083ada044d7e9ad09604742600ee7b/finetune/tagging/tagging_tasks.py#L144 Tagging task only contains the first segment and it should be zeros, right? @clarkkev

kamalkraj

pretrain with multigpu

3

Hi, guys, can we use multi-gpus to pretrain electra-small or electra-base now. Anyone figured it out? thanks

652994331

First of all, thank you for sharing great work ! I was wondering how would you recommend choosing optimal hyperparams for large batch size ? For example, if i train...

robinsongh381

On Page 13 of the paper, fine-tuning details part, the paper mentions that "we searched for the best number of train epochs out of [10, 3] for each task. For...

lgdgodv

NumPy Import Error

2

Whenever I try to use run_pretraining.py I get an error telling me that importing the NumPy C-extensions failed. I have already verified I am using the correct Python version and...

tommybean

no module named tensorflow.contrib

1

- python3.8 - transformers==4.4.1 - tensorflow==2.5.0 On running: ``` python run_pretraining.py --data-dir data --model-name electra_small ``` it exits with the following error: ``` from tensorflow.contrib import layers as contrib_layers ModuleNotFoundError:...

sprajagopal