Proposal for Model Reconstruction Based on the Latest Memformer Research Paper

[nemuihitojf]

@lucidrains

I am reaching out to suggest an update to the Memformer model, inspired by the recent advancements presented in the latest research paper about this model. The paper, published on 2022/04/12, offers new insights and methodologies that could be pivotal in enhancing the current Memformer implementation.

Integrating these latest findings could help keep the model at the forefront of the field, potentially improving both its performance and efficiency. For detailed information on the advancements, please refer to the research paper linked here: https://arxiv.org/abs/2010.06891.

Best regards,

[lucidrains]

what is new in the most recent paper that is not in the repo?