LLM4GEN
LLM4GEN copied to clipboard
Confused with eq(8)
Hi, thanks for your excellent work! In eq(7), ๐_tโฒ = Concat(๐ โ ๐_๐โฒ , ๐_๐ก ). In eq(8), ๐ฅ = CA(๐ฅ, ๐_๐กโฒ ). Thus x should be CA(๐ฅ, Concat(๐ โ ๐_๐โฒ , ๐_๐ก ) ). But in the paper, x in eq(8) is assigned as ๐ โ CA(๐ฅ, ๐_๐ ) + CA(๐ฅ, ๐_๐ก ). I have 2 questions:
- Why can the concat operation become an add operation?
- Why does ๐_๐โฒ become ๐_๐๏ผ
- How deep is your CAM?