nhatkhtn comments

Repositories
Issues
Comments

Results 2 comments of


                                            nhatkhtn

LlamaAttention forward function type hint is incorrect

Fixed with #38998

[Bug Report] RMSNormPre in Transformer_lens is maybe different from Llama source code?

@wangyifei0047 I believe you are mistaken. In the current version (2.15.4), the normalization layer used at the start of each layer for Llama is **RMS**, not RMSPre as you pointed...