Griffin icon indicating copy to clipboard operation
Griffin copied to clipboard

Published 20 hours ago • verified publisher icon kyegomez
→

Metadata

Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"

Results 1 Griffin issues
Sort by recently updated
recently updated
newest added

[BUG] Calculation of alpha and initialization of lambda incorrect

3 comment

Hello, I noticed two deviations from the Griffin paper in your code. # Lambda Here, Lambda is initialized as: https://github.com/kyegomez/Griffin/blob/83bbfdd9b0698cc27c19439ec16fb4fce07436c9/griffin_torch/main.py#L39-L42 However, the Griffin paper states in the second part of...

ozppupbg avatar ozppupbg

bug

← Metadata

41
Stars
3
Forks
Watchers

Owner

verified publisher icon kyegomez

Metadata

Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"

Back