LLM4GEN icon indicating copy to clipboard operation
LLM4GEN copied to clipboard

Confused with eq(8)

Open CharlesGong12 opened this issue 7 months ago โ€ข 2 comments

Hi, thanks for your excellent work! In eq(7), ๐‘_tโ€ฒ = Concat(๐œ† โˆ— ๐‘_๐‘™โ€ฒ , ๐‘_๐‘ก ). In eq(8), ๐‘ฅ = CA(๐‘ฅ, ๐‘_๐‘กโ€ฒ ). Thus x should be CA(๐‘ฅ, Concat(๐œ† โˆ— ๐‘_๐‘™โ€ฒ , ๐‘_๐‘ก ) ). But in the paper, x in eq(8) is assigned as ๐œ† โˆ— CA(๐‘ฅ, ๐‘_๐‘™ ) + CA(๐‘ฅ, ๐‘_๐‘ก ). I have 2 questions:

  1. Why can the concat operation become an add operation?
  2. Why does ๐‘_๐‘™โ€ฒ become ๐‘_๐‘™๏ผŸ
  3. How deep is your CAM?

CharlesGong12 avatar Jul 10 '24 13:07 CharlesGong12