mamba
mamba copied to clipboard
state-spaces
I test forward and step function with same input_ids and intermediate states. But the output logits and states are quite different. The code is like this: ```python for i in...
Compared to the method of using the CLS token provided by BERT to extract the entire sentence embedding, is Mamba's method of placing the CLS token effective? My intuition is...
ImportError: /anaconda3/lib/python3.11/site-packages/causal_conv1d_cuda.cpython-311-x86_64-linux-gnu.so: undefined symbol: _ZN3c107WarningC1ENS_7variantIJNS0_11UserWarningENS0_18DeprecationWarningEEEERKNS_14SoureLocationENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEb
First of all, thank you very much for your outstanding work. In my task, I successfully replaced the Mamba1 module and encountered the following problem during the backward of the...
Hi, Thank you for your awesome work! I have a query regarding the reproducibility of Mamba. Even though I use `torch.use_deterministic_algorithms(True, warn_only=False)` in my code, after adding Mamba, my code...
``` Traceback (most recent call last): File "test_mambav2.py", line 6, in from mamba_ssm import Mamba File "/home/test/miniconda3/envs/mamba/lib/python3.8/site-packages/mamba_ssm/__init__.py", line 5, in from mamba_ssm.modules.mamba2 import Mamba2 File "/home/test/miniconda3/envs/mamba/lib/python3.8/site-packages/mamba_ssm/modules/mamba2.py", line 23, in from...
My environment Package Version Editable project location ------------------------ ------------ --------------------------------------------------------- absl-py 2.1.02 addict 2.4.0 aiohttp 3.9.10 aiosignal 1.3.1ost2 alembic 1.13.0 asttokens 2.4.1 async-timeout 4.0.3 attrs 23.1.0 Automat 22.10.0 blinker 1.7.0...
Hello.I have been reading mamba-1 recently, and there is one issue that I do not quite understand。Based on the pseudo code in S6, the discretized A matrix has dimensions [B,...
Enable the mamba block to support variable-length sequence inputs using positional encoding. Passing Positional Indices results in negligible performance loss for the mamba block. For common variable-length sequence distributions, performance...
