bug: extremely slow on my VM since 0.4.4

[philippe-torteau-laposte-iteam]

Describe the bug since 0.4.4 version, jan.ai doesn't work fine on my VM.

Steps to reproduce since 0.4.4 version, jan.ai doesn't work fine on my VM. my VM is a 32 vCPU 64 Go RAM Windows 2019 server. it works really fine until 0.4.3 version. Now, jan starts but it's very slow to load model and use it. vCPU type is intel(R) Xeon(R) Gold 6448Y. i think the trouble comes from Nitro and the need of some instructions. any idea ? i try each version since 0.4.3, even the last one but it doesn't work any more even loading a 4GB model is very slow and reactivity to prompt is very very slow.

Expected behavior with 0.4.3 version loading model and answering speed are really fine. Loading a 25Go Model takes only 30 seconds

Environment details

• Operating System: Windows 2019 server
• Jan Version: [e.g., 0.4.3] 0.4.4 and above
• Processor: intel(R) Xeon(R) Gold 6448Y
• RAM: 64GB
• Any additional relevant hardware specifics: [e.g., Graphics card, SSD/HDD] no GPU

Logs no error in log, just ultra slow

Additional context i try several version of Nitro without any change ( 3.16, 3.21....). it works with nitro 2.6. i also try all the versions of nitro 3.15 without success

[Van-QA]

hi @philippe-torteau-laposte-iteam, would you mind retrying with our latest Jan release? https://jan.ai/

[philippe-torteau-laposte-iteam]

Hi, of course i tried it but it's the same behavior. it's still very very slow i have produced a report from CPUZ to analyse the caracteristics of my processor . I join it to the discuss WDNB4370.txt

[louis-jan]

Hi, of course i tried it but it's the same behavior. it's still very very slow i have produced a report from CPUZ to analyse the caracteristics of my processor . I join it to the discuss WDNB4370.txt

Yeah, this is a known issue. We updated the cpu_threads to be set to approximately the number of physical cores. However, the VM does not return an exact number of physical cores.

There is a hot-fix to allow users to override those settings by modifying model.json. That is not an ideal fix. We will work on a better approach. Please use that workaround for now.

[Van-QA]

Temporally closing this with the workaround 🙏

[philippe-torteau-laposte-iteam]

hi, sorry but changing this setting does nothing in my case. i modify this param in nitro.json without any succes also , il modify this param for one model to test and the jan behavior is the same. very very slow and CPU 100%