annotated_deep_learning_paper_implementations icon indicating copy to clipboard operation
annotated_deep_learning_paper_implementations copied to clipboard
verified publisher icon labmlai

Question about value_pe

Open Youngea opened this issue 1 year ago • 1 comments

In the file, rope/value_pe/init.py image

It seems the value embedding is rotated twice in line 230 and line 234. From my understanding, only one rotation is enough. Please let me know whether I misunderstood this. Thanks.

Youngea avatar May 07 '24 13:05 Youngea

Thanks. It's a bug

vpj avatar May 20 '24 08:05 vpj

Fixed it here https://github.com/labmlai/annotated_deep_learning_paper_implementations/commit/09d09379c2169eac06662e17cb9969dc6e48e36a

vpj avatar Jun 20 '24 07:06 vpj