Sparse LLM Inference

Open mmirman opened this issue 2 years ago • 4 comments

Jun 13 '23 19:06 mmirman

Hi @mmirman , I would like to work on this as well. Could you provide me some more context?

Jun 14 '23 09:06 Nike682631

I'm also interested. Are we talking about rolling out another entire inference engine here?

Jun 14 '23 18:06 ThePerfectComputer

@ThePerfectComputer @Nike682631 there's an active discussion in the discord's engineering channel about how to make this happen. Join there to discuss: https://discord.gg/qaFf7S373c

Jun 14 '23 18:06 mmirman

This is great! Any existing efforts (commercial or not) doing this well? Interesting to consider parallels to neuroscience.

Jun 18 '23 03:06 bmorphism