Artur A. Galstyan
Artur A. Galstyan
On this website: https://astro.build/integrations/official/ the integration box for Prism just links back to the main page instead of linking to this (for example): https://docs.astro.build/en/reference/api-reference/#prism-
This PR adds the Mamba Block. It's still in early draft modus. Along with the Mamba Block, the full Mamba model will be shown in the docs as an example.
Added RoPE embeddings from [the RoFormer paper](https://arxiv.org/pdf/2104.09864.pdf). I need to add this to my transformer to perform some tests first before I can mark this as ready. Also if it's...
Hi there, I had just translated the Mamba layer from [here](https://github.com/johnma2006/mamba-minimal) to [Equinox](https://github.com/Artur-Galstyan/kira/blob/mamba/kira/model/mamba.py). Would you accept a PR for this? PS: To get the most out of Mamba, we'd need...
While checking this PR #568, I noticed that the "process_heads" part actually shouldn't be part of the RoPE embeddings PR as it's a separate thing. In theory, you could process...
Hi, not sure if this is really a bug or intended, but it's not possible to pass an `eqx.Module` as carry in a `jax.lax.scan`. Here is the MVP: ```python import...
Hey Patrick, I was wondering if we could add RoPE and Sinusoidal embeddings to Equinox. It got a bit tedious to rewrite those every time, so I think it'd be...
Hi Patrick, a lot of issues are labelled "question". Perhaps it would be better to have a wiki to post these questions to leave the "issues" section for actual issues?...
Hi Patrick, I've incorporated the changes. It's not 100% done yet, but I'd especially like you to double check the error handling regarding the shapes in the MHA file.
I'm working on a little project that can ease PyTorch model conversion to your own JAX model Shameless advertisement and after setting all the weights, and biases and states, I...