exo
exo copied to clipboard
Support local and custom models
Issue https://github.com/exo-explore/exo/issues/165
Done:
- Interface changes for adding a model with HuggingFace model name
- UI logic: put HuggingFace model name in "add model" text area (e.g.
owner/modelName), click Add, select the model name in the dropdown, and start chatting.
- UI logic: put HuggingFace model name in "add model" text area (e.g.
- Loading & running model from HuggingFace
WIP:
- stream tokenizer decode minor bug
- model name persisting (will probably use localSession)
- support local models
This looks great.
What we'd really need here to get this merged is automatically sharding the model. The PyTorch implementation here does this automatically, you can see how they do it https://github.com/exo-explore/exo/pull/139 That way any language model can be supported without having to explicitly write a sharded model implementation.