exo
exo copied to clipboard
[BUG] Minimum model size is 4GB
Describe the bug
In the dashboard, the minimum model size is 4GB. This makes it impossible to attempt to load smaller models on devices 4GB of memory, and often makes it impractical to load them on medium (8-16GB) devices.
To Reproduce
Check the dashboard
Expected behavior
The model sizes for, say, Qwen3-0.6B-4bit should be ~0.3GB
Actual behavior
The model sizes for, say, Qwen3-0.6B-4bit are 4GB in the dashboard
Environment
uv run exo on main