How can I remove the limitation that the CPU only calls half of the cores?

[Aizizhu]

How can I remove the limitation that the CPU only calls half of the cores?

[rupeshs]

You can change it by setting an environment variable GGUF_THREADS=10

https://github.com/rupeshs/fastsdcpu/blob/ee0c2aba561f4fad6a747ef984477547b4a39741/src/constants.py#L24

[Aizizhu]

You can change it by setting an environment variable GGUF_THREADS=10

https://github.com/rupeshs/fastsdcpu/blob/ee0c2aba561f4fad6a747ef984477547b4a39741/src/constants.py#L24

Thank you, I have found this in the previous issues. I would like to ask how to modify openvino and lcm.

[maolei133]

You can change it by setting an environment variable GGUF_THREADS=10

https://github.com/rupeshs/fastsdcpu/blob/ee0c2aba561f4fad6a747ef984477547b4a39741/src/constants.py#L24

Thank you, My server's CPU has 96 cores, and by default, it should be using 48 cores. However, in practice, only 24 cores are being utilized. Setting the "GGUF_THREADS" environment variable also doesn't seem to have any effect.

[rupeshs]

You can change it by setting an environment variable GGUF_THREADS=10 https://github.com/rupeshs/fastsdcpu/blob/ee0c2aba561f4fad6a747ef984477547b4a39741/src/constants.py#L24

Thank you, My server's CPU has 96 cores, and by default, it should be using 48 cores. However, in practice, only 24 cores are being utilized. Setting the "GGUF_THREADS" environment variable also doesn't seem to have any effect.

which mode are you using? is it LCM-OpenVINO?

[maolei133]

You can change it by setting an environment variable GGUF_THREADS=10 https://github.com/rupeshs/fastsdcpu/blob/ee0c2aba561f4fad6a747ef984477547b4a39741/src/constants.py#L24

Thank you, My server's CPU has 96 cores, and by default, it should be using 48 cores. However, in practice, only 24 cores are being utilized. Setting the "GGUF_THREADS" environment variable also doesn't seem to have any effect.

which mode are you using? is it LCM-OpenVINO?

Yes LCM-OpenVINO, the model uses rupeshs/FLUX.1-schnell-openvino-int4