electra icon indicating copy to clipboard operation
electra copied to clipboard

add code for continuing pre-training from an ELECTRA checkpoint

Open tuvuumass opened this issue 4 years ago • 3 comments

  • add a variable init_checkpoint to configure_pretraining.py
  • add code for continuing pre-training from an ELECTRA checkpoint to run_pretraining.py
  • update README (instructions for continuing pre-training from an ELECTRA checkpoint + remove unnecessary stuff, i.e., "num_train_steps": 4010000 or "learning_rate": 2e-4 * (4e6 + 10000) / 10000, as we can specifify the actual "num_train_steps" (i.e., 10000 in this case) and learning rate (2e-4) now.

tuvuumass avatar Jun 16 '20 02:06 tuvuumass

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

:memo: Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

googlebot avatar Jun 16 '20 02:06 googlebot

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

@googlebot I signed it!

tuvuumass avatar Jun 16 '20 02:06 tuvuumass

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

googlebot avatar Jun 16 '20 02:06 googlebot