Using input if size torch.Size([200, 100])

[alqurri77]

trafficstars

My input is a batch of 200 and values of 100. (torch.Size([200, 100])) So my feature is just a 1-D vector.

In the example code, it says you set batch, length, and dim. But I have no length. How do I set Mamba in this case?

batch, length, dim = 2, 64, 16
x = torch.randn(batch, length, dim).to("cuda")
model = Mamba(
    # This module uses roughly 3 * expand * d_model^2 parameters
    d_model=dim, # Model dimension d_model
    d_state=16,  # SSM state expansion factor
    d_conv=4,    # Local convolution width
    expand=2,    # Block expansion factor
).to("cuda")

[tridao]

Mamba is a sequence-to-sequence layer, just like attention. If you don't have the length dimension (i.e. it's not a sequence) then Mamba is likely not a good choice.