exo icon indicating copy to clipboard operation
exo copied to clipboard

Add custom model support

Open AlexCheema opened this issue 1 month ago • 0 comments

Motivation

Users want to run HuggingFace models that aren't in the default model list. This PR adds the ability to add and remove custom models via the API and dashboard.

Changes

  • Add POST /models/custom endpoint for adding user-specified HuggingFace models
  • Add DELETE /models/{model_id} endpoint for removing custom models
  • Add model management UI to dashboard (add/remove models)
  • Support alternative hidden size field names in config.json parsing: n_embd (GPT-2), n_embed (BLOOM), d_model (OLMo/T5)
  • Refactor model cards into registry pattern with separate JSON card files

Why It Works

The custom model endpoint fetches metadata from HuggingFace (config.json for layer count and hidden size, safetensors index for model size) and adds the model to the registry. The robust config parsing handles the variety of field names used across different model architectures.

Test Plan

Manual Testing

  • Test adding custom models via POST /models/custom with various HuggingFace model IDs
  • Test removing models via DELETE /models/{model_id}
  • Test dashboard model management UI

Automated Testing

  • All 154 existing tests pass
  • Type checking (basedpyright) passes with 0 errors
  • Linting (ruff) passes

🤖 Generated with Claude Code

AlexCheema avatar Jan 18 '26 12:01 AlexCheema