golgotha icon indicating copy to clipboard operation
golgotha copied to clipboard

Newly introduced architectures of transformers

Open AnselmJeong opened this issue 4 years ago • 1 comments

Do you happen to have any plans to implement newly introduced architectures in the near future? You mentioned 10 supported architectures, but Huggingface enlarged their repertoire of supported architectures to be at least 17. I'm personally interested in BART, T5 and ELECTRA...

I'm not complaining about this useful package, it is just a personal wish...

Thanks.

AnselmJeong avatar Apr 10 '20 02:04 AnselmJeong

If you want to play with other architectures the best is to create a pull request. The relevant parts where you need to make changes are

  • Adding extra lines for models here: https://github.com/bnosac/golgotha/blob/master/inst/python/BERT.py#L6
  • Referrring to these extra models with the names that you used in BERT.py here: https://github.com/bnosac/golgotha/blob/master/R/AAA.R#L19
  • Make sure the package uses the correct version of transformers at https://github.com/bnosac/golgotha/blob/master/DESCRIPTION#L21
  • Add unit tests to make sure your change works in https://github.com/bnosac/golgotha/tree/master/inst/tinytest

jwijffels avatar Apr 10 '20 08:04 jwijffels