Dr. sc. Matthew Mirman comments

Results 46 comments of


                                            Dr. sc. Matthew Mirman

Implement 4bit, 8bit quantization for Nvidia GPUs

If this isn't done with a library its a $200 ticket, if so its a SWAG ticket

Custom batch decoding algorithm

@rishirishhh you are welcome to work on it! Getting assigned tickets officially requires contributor status - which you can achieve by winning bounties!

Ability to choose what model to use for data synthesizer

Before we can fix this we need to: 1. make sure the model can do data synthesis, and set it up 2. document which models are good at what with...

Implement 4bit, 8bit quantization for all hardware

AMD, FPGA... ALL! (shaders?)

Implement 4bit, 8bit quantization for all hardware

($400 provided its not solved by a library call)

Add memory state management for LLMs

How do we measure success here?

Web playground 1: Develop a web playground for LLM VM

Be warned! We have one internally, chat.dev and we want to open-source it

Web playground 1: Develop a web playground for LLM VM

Warning! It'll have to be better than the playground we already have :-)

Sparse LLM Inference

@ThePerfectComputer @Nike682631 there's an active discussion in the discord's engineering channel about how to make this happen. Join there to discuss: https://discord.gg/qaFf7S373c

Implement checkpoint management for storing training states

We kind of already have this, but we need a way to manage checkpointing too!