flash-fft-conv icon indicating copy to clipboard operation
flash-fft-conv copied to clipboard

FlashFFTConv can be definitely be implemented on Mamba, right?

Open nam-drun opened this issue 1 year ago • 2 comments

nam-drun avatar Mar 06 '24 12:03 nam-drun

Mamba does not have a convolutional form, so there isn't an exact mapping. For Mamba you'll have to use the scan formulation as documented in the paper.

DanFu09 avatar Mar 06 '24 18:03 DanFu09

For Mamba you'll have to use the scan formulation as documented in the paper. I get what you mean (algorithm 2 in figure 2). I was confused a bit because figure 3 showing mamba architecture also has Conv layer???

nam-drun avatar Mar 07 '24 09:03 nam-drun