transformer-models

transformer-models copied to clipboard

when i try to run mdl = bert It stays an infinite time running. When I open the file that it creates on the computer "Parameters" it cannot be opened

1

when i try to run mdl = bert It stays an infinite time running. When I open the file that it creates on the computer "Parameters" it cannot be opened

MikelGuna

How to train a BERT model from scratch

6

How can I train a BERT model from scratch?

dwjung1

Add a BERT training example

1

I have opened this issue so that everyone can post which example they would like to use.

misataguchi

Add a GPT-2 training example

2

We would like to use these issues to gauge user interest. It is possible to use the GPT-2 implementation for further language model training. There is no example demonstrating this...

bwdGitHub

See [CircleCI](https://app.circleci.com/pipelines/github/matlab-deep-learning/transformer-models/43/workflows/6d79f2e1-d34c-45f0-9dec-4fe59b62a955/jobs/42) We can update the error ID we're testing against, or add extra validation for `bert.model`.

bwdGitHub

bert model function m-help should explain what "parameters" input is

1

vpapanasta

We would like to use these issues to gauge user interest. The BERT tokenizer is intended as an identical reimplementation of the original BERT tokenization. However it is possible to...

bwdGitHub

Do have any examples of "vision transformer"?

2

In addition to the NLP applications, what about the CV applications? Are there any examples of transformer being used for "object detection", "behaviour recognition" or even "image classification"? It would...

cuixing158

We would like to use these issues to gauge user interest. The GPT-2 implementation does not include dropout layers. This would be useful for further pre-training and fine-tuning workflows to...

bwdGitHub

We would like to use these issues to gauge user interest. Sparse cross entropy allows the computation of cross entropy loss without one-hot encoding of the target class. This is...

bwdGitHub