mamba icon indicating copy to clipboard operation
mamba copied to clipboard

Passing an initial_conv_state in mamba_split_conv1d_scan_combined?

Open h-zhao1997 opened this issue 7 months ago • 2 comments

Thank you for your outstanding work! I'm curious if you've thought about including an additional parameter in the mamba_split_conv1d_scan_combined function to accept an initial_conv_state. This could open up some intriguing applications, like treating initial_conv_state as a trainable parameter. I've observed that Mamba2 has already implemented the ability to pass initial_states for the SSM layer. In your opinion, would it be beneficial to adopt a similar strategy for the 1D convolution layer?

h-zhao1997 avatar Jul 12 '24 05:07 h-zhao1997