exo
exo copied to clipboard
Add custom model support
Motivation
Users want to run HuggingFace models that aren't in the default model list. This PR adds the ability to add and remove custom models via the API and dashboard.
Changes
- Add
POST /models/customendpoint for adding user-specified HuggingFace models - Add
DELETE /models/{model_id}endpoint for removing custom models - Add model management UI to dashboard (add/remove models)
- Support alternative hidden size field names in config.json parsing:
n_embd(GPT-2),n_embed(BLOOM),d_model(OLMo/T5) - Refactor model cards into registry pattern with separate JSON card files
Why It Works
The custom model endpoint fetches metadata from HuggingFace (config.json for layer count and hidden size, safetensors index for model size) and adds the model to the registry. The robust config parsing handles the variety of field names used across different model architectures.
Test Plan
Manual Testing
- Test adding custom models via
POST /models/customwith various HuggingFace model IDs - Test removing models via
DELETE /models/{model_id} - Test dashboard model management UI
Automated Testing
- All 154 existing tests pass
- Type checking (basedpyright) passes with 0 errors
- Linting (ruff) passes
🤖 Generated with Claude Code