exo icon indicating copy to clipboard operation
exo copied to clipboard

[BUG] Minimum model size is 4GB

Open Evanev7 opened this issue 2 days ago • 0 comments

Describe the bug

In the dashboard, the minimum model size is 4GB. This makes it impossible to attempt to load smaller models on devices 4GB of memory, and often makes it impractical to load them on medium (8-16GB) devices.

To Reproduce

Check the dashboard

Expected behavior

The model sizes for, say, Qwen3-0.6B-4bit should be ~0.3GB

Actual behavior

The model sizes for, say, Qwen3-0.6B-4bit are 4GB in the dashboard

Environment

uv run exo on main

Evanev7 avatar Dec 23 '25 12:12 Evanev7