Misleading sub-division naming, can you clarify please?

[Ibrokhimsadikov]

I am a bit confused the categorization of LLMs into Encoder only, Decoder only, Encoder-Decoder. Finding a bit hard time to understand what these terms actually mean:

Yann Lecun posted that on twitter: https://twitter.com/ylecun/status/1651762787373428736?lang=en

Can you please shed some light?

Thanks

[Vincent-Stragier]

@Ibrokhimsadikov,

It refers to the modification of the transformer architecture. Generative Pretrained Transformer are decoder only models, etc.

See https://arxiv.org/abs/1706.03762. If you really want to learn LLMs in depth, you can begin with these references: https://gist.github.com/rain-1/eebd5e5eb2784feecf450324e3341c8d