mamba

mamba copied to clipboard

why not try 7B or more?

1

af-74413592

Hello, I would like to perform single-step inference using Mamba, which means my inference task only needs to generate one token (or extract the last token embedding, without needing intermediate...

41924076

Consider allowing hidden state initialisation via ssm_state input parameter for selective_scan_fn

5

Please, please, consider adding the ssm_state input parameter for selective_scan_fn to allow hidden state initialisation for the Mamba block. Also please consider making hidden state differentiable as currently at selective_scan_fn...

govorunov

Hello, thanks for sharing your great work! I meet some problems when trying to understand the source code in 'selective_scan_interface.py'. I wonder what is the difference between 'SelectiveScanFn' and 'MambaInnerFn'?...

Charlie839242

MAMBA for Classification

2

First of all, it is excellent work. I want to do simple classification with the MNIST dataset, I tried many things but could not compile the model. How should I...

7unahan

How do I use mamba on windows?

3

Thanks for your work! I wonder if this code will run on windows?

dadadalizi146

Test failure when dim is large

1

Hi, I'm running the mamba test_selective_scan.py benchmark with increasing the model dimension and the tests starts to fail. Here is how I increase the dimension: ``` diff --git a/tests/ops/test_selective_scan.py b/tests/ops/test_selective_scan.py...

bilgeacun

I have tried to replace the self attention layer with mamba and hyena, but witness a worse performance for mamba. I am not sure whether it's because of my mal-setting...

multydoffer

selective_scan_cuda error

3

I'm using the m1 chip version of MacOS and python3.10 pytorch2.2.1 natively tried to use mamba_ssm.ops.selective_scan_interface native, so I tried to skip here, the truth is that it works, and...

wang935415150

CrossAttention To CrossMamba

4

Dears Authors! Thank you for your great work! I have a question about adapt from cross attention to cross mamba. Can i modify mamba from this  to this (with...

Kiet0712