exo icon indicating copy to clipboard operation
exo copied to clipboard

Emit download progress on start, and change downloads to be keyed by model_id

Open AlexCheema opened this issue 2 months ago • 0 comments

Motivation

We added a download page to the dashboard which shows the currently download status of each model on each node. Users have reported this to be extremely useful.

However, we don't currently fetch the download progress on start, so it doesn't show any model's download status.

Changes

Fetch and emit model download status on start of worker, and periodically every 5 mins. Also to support this, I changed download_status to be keyed by model_id instead of shard, since we want download_status of each model, not each shard.

Why It Works

The dashboard already implements the correct functionality, we just weren't populating the download status in the state. Now it gets populated and shows correctly.

Test Plan

Manual Testing

On a cluster of 2 x 512GB M3 Ultra Mac Studio, I launched an instance onto one node that hadn't been downloaded. I checked the download page and it showed the in progress download. I downloaded it to completion, restarted exo on both nodes, and then opened the download page and it showed the model as 100% downloaded and other models as 0% that hadn't been downloaded.

AlexCheema avatar Dec 30 '25 02:12 AlexCheema