exo-explore
Show downloaded models, improve error handling, ability to delete models, side bar with more detail, button to go back to chat history
I wanted a way to show which models have already been downloaded. Ultimately, I would like to add the ability to manage those models such as deleting them. Might make more sense in a sidebar where you chose the model and can manage it.
I also added more robust handling of errors in index.html to safely access them as I saw some warnings in the main branch when no errors were set.
In index.js I combined error handling into a single flow for the populateSelector method using response.json() instead of manually parsing. I also added a helper method to set the error to reduce duplicate code. Something I should have done with my last PR.
Open to suggestions on these changes.
I am seeing now some of the robust error handling was intentional and merged in here. I can change back if that is preferred.
+1 on the concept, but this is an issue:
Oh wait, that's in Safari. It actually works on Chrome and Firefox. Still, should probably fix before merge.
+1 on the concept, but this is an issue:
Oh wait, that's in Safari. It actually works on Chrome and Firefox. Still, should probably fix before merge.
I fixed these issue in https://github.com/cadenmackenzie/exo/pull/1/files which you can merge into this PR
Awesome!
Got it working with the suggested changes. Want to do some testing in the morning then will update this PR.
Added new abstract method in ShardDownloader, implemented get_shard_download_status in HFShardDownloader leaning on get_local_snapshot_dir, get_weight_map, get_allow_patterns helper functions and then checks the percentage of model downloaded for models where local files are found. Removed old function for checking percentage.
Currently shows "downloaded" for fully downloaded models and "X% downloaded" for models not fully downloaded in dropdown.
I don't love how this is being refreshed using the modelPoolInterval because it lags a little for models that are actively being downloaded so might work on how to improve that.
I think it could be worth moving this logic as well as active downloads to a sidebar. I like how active downloads are being shown in the chat when initiated but if we moved that to a sidebar, it could be a centralized place to view all models, choose a model, see activity of downloaded models, and being able to remove local downloads to free up space. Similar to how you can remove local models in LM Studio.
Open to suggestions.
Found an issue in handle_model_support that was creating HFShardDownloader without quick_check=true so it was starting download of models when being checked for download percentage.
Will work on get_shard_download_status refactor
Hi @AlexCheema can you give me some more insight into what you want to be refactored? I initially thought we could reuse some of the download percentage logic that is happening during download but as far as I can tell, that is only checking against the remote during download in download_file(). Would you want to pull some of that logic out into another helper function to use in get_shard_download_status() or am I missing something?
Hi @AlexCheema can you give me some more insight into what you want to be refactored? I initially thought we could reuse some of the download percentage logic that is happening during download but as far as I can tell, that is only checking against the remote during download in download_file(). Would you want to pull some of that logic out into another helper function to use in get_shard_download_status() or am I missing something?
Yeah I think pulling some of that logic out would make sense. I think it's in general in need of a good refactor if you want to take a go at that and will bump up the bounty to $300.
Hi Alex, please review. I didn't modify download_file much but added a check using the new helper method.
Modified the get_shard_download_status to lean on the helper methods to calculate the overall percentage of files and return that. Also updated chatgpt_api to use that overall instead doing percent calculation there. Should fix the pattern matching issue that you identified as well.
Did you test this after the merge? Models aren't loading and getting syntax errors.
Hi @AlexCheema , my apologies. I did test it and it was working for me even without the inference_engine_classes defined. Not sure why.
I fixed it and installed Pylance.
@AlexCheema merged in side bar changes and delete functionality and added functionality to use server sent events to show each model as its checked. Also shows loading icon in side bar until the first model is loaded. Quick demo
When I delete a model, the following error appears: An unknown error occurred
It does actually delete the model however tinychat shows this error.
Trying to reproduce that error but I can't seem it do it. Can you send me your steps? What line is that error being thrown on?
Okay, I can't reproduce now either so that's fine.
One more thing: could you add something that indicates whether a download is currently in progress? Sometimes a download could fail and you don't realise it or it could be in progress but got stuck. An indication of whether exo is actively trying to download a shard would be super helpful for debugging. @cadenmackenzie
@AlexCheema I added a spinner animation to the model in the sidebar that is actively being downloaded. I can modify that to make it a progress bar or say "Downloading" if that is better. Demo here. Is that what you were envisioning?
Merge conflicts from main branch resolved.
@AlexCheema I added a spinner animation to the model in the sidebar that is actively being downloaded. I can modify that to make it a progress bar or say "Downloading" if that is better. Demo here. Is that what you were envisioning?
Merge conflicts from main branch resolved.
I noticed this is being done from the frontend right now. Is there any way to use the backend as the source of truth to check if a model download is in progress instead? This will be incredibly useful for debugging e.g. if I'm using the API and I want to check what model downloads are in progress.
@AlexCheema right now the frontend is matching what is being returned by /v1/download/progress and /modelpool to determine if a model download is in progress on the sidebar. do we want to try to combine some of that logic into one endpoint? Or add a check to /modelpool to return true or false if the model is being downloaded. /v1/download/progress is the current source of truth for checking if a download is in progress and polls every second when a download is in progress.
@AlexCheema right now the frontend is matching what is being returned by
/v1/download/progressand/modelpoolto determine if a model download is in progress on the sidebar. do we want to try to combine some of that logic into one endpoint? Or add a check to/modelpoolto return true or false if the model is being downloaded./v1/download/progressis the current source of truth for checking if a download is in progress and polls every second when a download is in progress.
That works! I just wanted to make sure the source of truth for this was the server, not the frontend.
can you clarify what you mean? Do you want the logic to be combined into one endpoint or are you okay that the server is the source of truth using the two endpoints and the matching of results on the frontend as it is right now? @AlexCheema
can you clarify what you mean? Do you want the logic to be combined into one endpoint or are you okay that the server is the source of truth using the two endpoints and the matching of results on the frontend as it is right now? @AlexCheema
Unifying into one endpoint would be better.
I also added an endpoint to trigger a download here: https://github.com/exo-explore/exo/pull/526 It would be great if that could be triggered from the UI.
@AlexCheema I got a working version of the endpoints combined but I have some concerns. If the endpoints are combined, the updates for the Download Progress Section rely on a response from /modelpool which is called every 5 seconds and checks the status of every model. Ultimately these feel like processes that should be kept in their own endpoints because /modelpool is checking each local model against the remote whereas /download/progress is using the node.node_download_progress during a download. If anything I think we could use the responses from /download/progress to update the sidebar for the model being downloaded so it is more real-time. Let me know if I am missing something as to why you would want these combined into one endpoint.
Also added the trigger to start a download for partially downloaded or not downloaded models using the download endpoint. Demo here. @AlexCheema