The Reproducibility of Mamba

[Yisen-Feng]

trafficstars

Hi,

Thank you for your awesome work! I have a query regarding the reproducibility of Mamba. Even though I use torch.use_deterministic_algorithms(True, warn_only=False) in my code, after adding Mamba, my code lost reproducibility. I wonder if it is possible to have a deterministic Mamba.

[GuHY777]

I also have met this problem!

[tridao]

The current backward pass is not deterministic (it uses atomic adds).

[tridao]

I think that's the pytorch cumsum, so it's out of the scope of this repo.

[NLHuang]

I think that's the pytorch cumsum, so it's out of the scope of this repo.

I see. Thank you.