memorizing-transformers-pytorch icon indicating copy to clipboard operation
memorizing-transformers-pytorch copied to clipboard

Published 20 hours ago • verified publisher icon lucidrains

is it a t5 arch or decoder only gpt style arch?

Open brando90 opened this issue 2 years ago • 1 comments

brando90 avatar Feb 10 '23 20:02 brando90

T5 is also a decoder-only architecture. The paper uses a decoder-only transformer which this memorizing transformer also seems to be!

Jayant1234 avatar May 31 '23 22:05 Jayant1234