mamba
mamba copied to clipboard
Issue: Forward pass of mamba block results in 0s on a Quadro P6000 GPU
Once I had my mamba environment set up and mamba-ssm installed successfully, I ran the example code provided by the repository and the output of x and y was as follows: tensor([[[-1.9903e+00, -9.2922e-01, -2.8890e+00, ..., -2.4487e-03, -1.4836e+00, -1.7656e+00], [-6.2470e-01, -4.3941e-01, 1.2152e+00, ..., -1.2207e-01, -2.0163e-02, -2.1134e+00], [-1.2493e+00, -4.5132e-01, 6.3297e-01, ..., -1.6900e+00, -1.4243e+00, -4.9173e-01], ..., [ 1.1544e-01, -1.4427e+00, -3.0226e-01, ..., 7.3694e-01, -3.7878e-01, 2.1897e+00], [ 6.6608e-01, -2.5774e-01, -9.2452e-01, ..., -2.6071e-02, 1.2988e+00, -1.8465e+00], [-7.5659e-01, -1.3786e+00, -8.3231e-01, ..., 9.0802e-01, -1.7450e-01, -1.8339e+00]],
[[ 6.3811e-01, -2.0693e+00, -8.9093e-01, ..., 1.8312e-01,
1.1453e-01, 4.1747e-01],
[-1.1871e+00, 2.0305e-01, 5.3381e-01, ..., -2.3646e-01,
1.2816e+00, -5.6714e-01],
[-1.6325e+00, -1.0873e+00, -4.7420e-01, ..., 1.3370e+00,
2.8081e-02, -9.4508e-02],
...,
[-3.5926e-01, -4.8741e-01, -1.1013e+00, ..., -2.1140e-01,
-3.3222e-01, 3.1988e-01],
[-1.0671e+00, -3.9777e-02, 1.3775e+00, ..., 4.9738e-01,
4.1687e-01, -4.6370e-01],
[-6.4432e-01, -1.6179e+00, -2.2629e-01, ..., 8.0204e-01,
1.1772e+00, -3.3466e-02]]], device='cuda:0')
tensor([[[0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], ..., [0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.]],
[[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.]]], device='cuda:0',
grad_fn=<MambaInnerFnBackward>)
Is there anyone who has encountered the same problem with the output of 0? My GPU is a Quadro P6000.