Thomas Gauthier-Caron

I'm having the same issue. @NotAFile did you find a workaround?

I had the same problem. Seems like it has to do with GCC. I was able to successfully compile it by downgrading gcc-arm-embedded from 7-2017-q4 to 5.4. To do it...

I think that this would be a useful feature. For example preserving wikilinks would enable linking to english glosses for foreign words that do not contain a full definition (e.g....

I added support for this in PR #158

I'm also experiencing this on Arch Linux with Gnome 43.3 and LibreWolf 110. It seems to work fine in Chromium. Edit: Managed to get it working. LibreWolf needs some [tweaks](https://librewolf.net/docs/faq/#how-do-i-get-native-messaging-to-work)...

I implemented something in this direction using singular value decomposition (SVD). I call it [LoRD](https://github.com/thomasgauthier/LoRD) for *Lo*w-*R*ank *D*ecomposition

@abetlen I'm also curious to know if this is still a planned feature. Thank you

Also experiencing this with `meta-llama/Meta-Llama-3-8B-Instruct`, this makes the library more or less unusable for me. Which is a shame because I love sglang.

@jukofyork I have been playing around with merging the 8x22B experts into a dense model. So far the best results I had were using the softmaxed router logits over [exllamav2...

Yes true, the technique could be useful for creating MoEs too, didn't think of that! To answer your question, I'm actually doing the softmax per token per layer (same as...