janhq
bug: "Recommended" labels don't autoupdate when user toggles CPU and GPU modes
Bug Description By default Jan will choose CPU mode. My local AI models are marked as "Recommended". When enabling Experimental Mode and Vulkan support, all local models are marked as "Slow on your device" (probably due to my GPU's 4GB of VRAM). After disabling Vulkan support and Experimental Mode (returning to the default settings), all local models will still appear as "Slow on your device" and will fail to start.
Steps to reproduce Local models appear as "Recommended"
- Open Jan Settings
- Open the Advanced Settings tab
- Enable Experimental Mode and Vulkan Support
Local models will now appear as "Slow on your device"
- Disable Experimental Mode and Vulkan Support
Local models will still be marked as "Slow on your device" despite having returned to the default settings.
Expected behavior After disabling Experimental and Vulkan settings, all local models will return to the previous state and appear as "Recommended" (if they ever did in the first place).
Environment details
- Operating System: Windows 11 23H2
- Jan Version: v0.4.10
- Processor: AMD Ryzen 5 3600
- RAM: 32GB
- GPU: RX570 4GB
Solution Settings > Advanced Settings > Reset to Factory Default Then, you can import your models again and they will run using the default CPU mode (just like it did at the start, before changing any Experimental or Vulkan settings), and they will be marked as "Recommended".
Assigning this to @hiento09 since we are still investigating how to display the Memory value when using Vulkan support 🙏
Adding the script from contributor for future reference: https://discord.com/channels/1107178041848909847/1213048058955300934/1243533268620541954
Should be solved once we complete https://github.com/janhq/cortex.cpp/issues/1108
I'll mark this as duplicate & should be a user story of the janhq/cortex.cpp#1108 epic
Handling this bug as part of https://github.com/janhq/cortex.cpp/issues/1165
This is an app bug where it does not reset other settings when the experimental feature is toggled off.
Scenarios:
- Users toggle ON the Experimental Settings & Vulkan Support
- Users toggle OFF the Experimental Settings. The app should toggle off others automatically.
Bug fix: It should toggle off features gated by Experimental Settings when toggling off.
Regarding the issue where app could not get Vulkan device VRAM which is not scoped by the issue, it should be a part of #1165
all local models will still appear as "Slow on your device"
This is an app bug where it does not reset other settings when the experimental feature is toggled off.
Scenarios:
- Users toggle ON the Experimental Settings & Vulkan Support
- Users toggle OFF the Experimental Settings. The app should toggle off others automatically.
Bug fix: It should toggle off features gated by Experimental Settings when toggling off.
Regarding the issue where app could not get Vulkan device VRAM which is not scoped by the issue, it should be a part of #1165
all local models will still appear as "Slow on your device"
@louis-jan I think we should handle this as part of a larger sprint on Hardware Detection, "llama.cpp version switching":
- https://github.com/janhq/cortex.cpp/issues/1218
- https://github.com/janhq/cortex.cpp/issues/1165
This is a combination of 2 issues
- bug: app bug where it does not reset other settings when the experimental feature is toggled off (Louis' reply 2 message above)
- feature request/epic: Support Vulkan devices
@louis-jan can I check if the app bug is pending or resolved? (and tag which issue)
I propose we use this issue for the app bug. Then separately track the Vulkan support feature request under #1165 and #1218
