Titans
Titans copied to clipboard
[FEATURE] Settings related to common layers.
Describe the feature
Compared to vanilla PyTorch, Titans right now includes many unnecessary codes for example multiple files for MLP.
We could provide common MLP, Attention ... modules for users to avoid redundancy in both our repo and users' codes.
I have attached an simple example of PyTorch code as well.