[BUG] Have you checked the correctness of the code?

[wzm2256]

In the file https://github.com/kyegomez/MegaVIT/blob/main/mega_vit/main.py

1. In line 150, you mentione "MegaVit model from https://arxiv.org/abs/2106.14759", but that paper is "The Eisenbud-Green-Harris Conjecture". I do not think this is the correct paper.
2. In line 140, all input is normalized, however, in line 85, the input is normalized again. That means all input of the attention layer is normalized twice. This seems to be a bug.

Upvote & Fund

• We're using Polar.sh so you can upvote and help fund this issue.
• We receive the funding once the issue is completed & confirmed by you.
• Thank you in advance for helping prioritize & fund our backlog.