How to change mamba to mamba2

Open xiaogege1210 opened this issue 1 year ago • 2 comments

Is it just a matter of changing the version of mamba_ssm from 1.x to 2.x？

Sep 12 '24 07:09 xiaogege1210

Or would upgrading mamba_ssm from 1.x to 2.x change anything?

Sep 12 '24 07:09 xiaogege1210

it's just about picking which model to import from https://github.com/state-spaces/mamba/tree/main/mamba_ssm/modules

however if you're using the MixerModel class, add ssm_cfg={"layer": "Mamba2"} as an argument

Sep 13 '24 15:09 maximilienleclei