tensor2tensor icon indicating copy to clipboard operation
tensor2tensor copied to clipboard

What are the transformers that is obviously better than transformer-2017?

Open guotong1988 opened this issue 3 years ago • 1 comments

The transformer-2017 is described in the paper 《attention is all you need》. Now 3 years past. Are there any transformers that surpass transformer-2017 in most of the task? Speedup without accuracy decreasing is also considered as a better one.

guotong1988 avatar Jan 20 '21 06:01 guotong1988

GPT3

GT135813 avatar Jan 20 '21 07:01 GT135813