mamba

mamba copied to clipboard

Make a wish for a docker image to run mamba.

2

cilinyan

How to weights merge?

5

I trained 3 models, but after averaging the weights, the model output is garbled!

junphine

Question about exporting Mamba models to GGUF format for Ollama deployment

1

I am interested in deploying mamba within the Ollama framework. I was wondering if thers's a way to export Mamba models to GGUF format, or if there are any recommended...

zhaohm14

Is Context Length dependent on training data's context?

6

I notice that passkey retrieval works well up to around 3-4k tokens. After that, it doesn't. That wasn't my intuition for SSMs, but I guess context length is still related...

RonanKMcGovern

AssertionError when training mamba model on my own dataset

2

Hi there! I want to train a mamba language model from scratch on my own dataset. However, during the training of process, there is a problem. When I set CUDA_LAUNCH_BLOCKING=1...

xianyu0623

Building wheel for causal-conv1d (setup.py) ...

7

为什么他不动了

zhangjie77

mamba-ssm for macos with M1

1

I understand that currently mamba requires nvcc (Cuda) to install. Is there a version planned for MacOS M1? pytorch already support the MPS and it shall be nice to have...

freedom-cognit

[Feature] Support variable-length sequences for mamba block

40

Support variable-length sequences for mamba block via `cu_seqlens` in the `forward` pass and `backward` pass, similar to what has been done (such as cumulative sequences `cu_seqlens` or lower triangular block...

zigzagcai

a problem when running "CAUSAL_CONV1D_FORCE_BUILD=TRUE pip install ."

5

Processing /home/ubuntu/causal-conv1d Preparing metadata (setup.py) ... error error: subprocess-exited-with-error × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [11 lines of output] Traceback (most recent call...

Eraseryl

TypeError: Mamba.__init__() got an unexpected keyword argument 'bimamba_type'

1

my envs: causal-conv1d 1.1.1 mamba-ssm 1.2.0.post1 but I installed it inside Windows，i do not know why

Cedar3