s4
s4 copied to clipboard
state-spaces
Structured state space sequence models
Dear authors/contributors, First of all, thank you so much for publishing such a great work. I think it is really inspirational and we will see this model (or its variants)...
Breathtaking work, absolutely amazing application of linear algebra. Beautiful. A few questions. Appendix of [S4 article](https://arxiv.org/pdf/2111.00396.pdf) mentions "... implementation of S4 uses the naive O(NL) algorithm ..._ " and [README.md](https://github.com/HazyResearch/state-spaces/blob/main/README.md)...
Hey, I'm trying to use the forward_state function. From time to time, I get this error: ``` RuntimeError: CUDA error: an illegal memory access was encountered CUDA kernel errors might...
Hello Everyone, I'm looking to use the standalone S4D replacement layer [here](https://github.com/HazyResearch/state-spaces/blob/main/src/models/sequence/ss/standalone/s4d.py) as a drop-in replacement to the attention mechanism in a transformer model. I'm wondering what is the best...
There exist various references to `src/models/sequence/ss/standalone/s4.py`, but such file does not exist. It would be lovely to either have a standalone version, or an installable version
Hey! I'm attempting to integrate the Sashimi Backbone into some audio models -- I'd like to train in convolutional mode and run _validation_ inference in RNN mode, but my reading...
I'm trying to reproduce experiments but the code is retuning a KeyError 'nvrtc' and the warning `[src.models.sequence.ss.kernel][WARNING] - CUDA extension for Cauchy multiplication not found` still appearing. Otherwise, I'm getting...
I'm currently writing a recurrent reinforcement library, with LSTMs, linear attention, etc that I would like to add S4 to. Unfortunately, I find S4D unable to learn in even simple...
When I was running with the command `python -X faulthandler -m train pipeline=mnist model=s4`, I found that there's a segmentation fault. I located the problematic code with: ``` File "/home/liluo/code/s4/src/models/hippo/hippo.py",...
Thanks for your amazing work! I have a question about the latent states of S4. S4 is based on SSM, in which x(t) 's latent dynamics are explicitly modeled. S4...
