Vim icon indicating copy to clipboard operation
Vim copied to clipboard

Mamba module initialization

Open mingtan2 opened this issue 1 year ago • 0 comments

Dear authors,

Thanks for the great work and releasing the codebase! I have one question regarding the Mamba module initialization.

From the code, I see its linear layers are initialized here and here, then the weights and biases are overwritten here and here, which zeros out the mamba linear layers' biases. So this line actually can be deleted? Please share your insights if it's intended. Thanks!

mingtan2 avatar Feb 05 '24 01:02 mingtan2