jan
jan copied to clipboard
feat: Models in Hub has recommendation labels based on user RAM
Context: It's part of hub revamp but kinda urgent so we want to separate & push it out first @urmauur
Todo
Add compatibility tag, similar to this but you decide where to put on the current Hub (not the hub design revamp - this is more urgent):
From @tikikun There are 3 values:
A. Minimum RAM for Model to run B. Amount of RAM that it's available to run any model C. Max RAM on user device
A = model_size * 1.25 (the multiplier needs to be defined - I will check for gguf note on his repo) If (A < C && A > B) => Warn user that they should turn off some other programs to get more B, then retry If (A>C) => Disable download/ run model - this saves users from freezing their machine If (A < B) => Run normally
We also needs to check the VRAM GPU as well, but I think we can do it in subsequent PR. It's similar to normal RAM calculation, but needs nested conditionals checks