jan icon indicating copy to clipboard operation
jan copied to clipboard

feat: Models in Hub has recommendation labels based on user RAM

Open 0xSage opened this issue 6 months ago • 2 comments

Context: It's part of hub revamp but kinda urgent so we want to separate & push it out first @urmauur

Todo

Add compatibility tag, similar to this but you decide where to put on the current Hub (not the hub design revamp - this is more urgent): Hub default - filter

From @tikikun There are 3 values:

A. Minimum RAM for Model to run B. Amount of RAM that it's available to run any model C. Max RAM on user device

A = model_size * 1.25 (the multiplier needs to be defined - I will check for gguf note on his repo) If (A < C && A > B) => Warn user that they should turn off some other programs to get more B, then retry If (A>C) => Disable download/ run model - this saves users from freezing their machine If (A < B) => Run normally

We also needs to check the VRAM GPU as well, but I think we can do it in subsequent PR. It's similar to normal RAM calculation, but needs nested conditionals checks

0xSage avatar Jan 05 '24 08:01 0xSage