fail HQQ on win10

[kalle07]

Describe the bug

updated to newest version on main load mixtralmodel mobiuslabsgmbh_Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-2bit-HQQ and get this error 22:52:36-341318 INFO Loading mobiuslabsgmbh_Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-2bit-HQQ 22:52:36-341318 ERROR Failed to load the model. Traceback (most recent call last): File "e:\text-generation-webui\modules\ui_model_menu.py", line 213, in load_model_wrapper shared.model, shared.tokenizer = load_model(selected_model, loader) File "e:\text-generation-webui\modules\models.py", line 87, in load_model output = load_func_maploader File "e:\text-generation-webui\modules\models.py", line 393, in HQQ_loader from hqq.core.quantize import HQQBackend, HQQLinear File "e:\text-generation-webui\installer_files\env\lib\site-packages\hqq\core\quantize.py", line 238, in class HQQLinear(torch.nn.Module): File "e:\text-generation-webui\installer_files\env\lib\site-packages\hqq\core\quantize.py", line 336, in HQQLinear def matmul_compile(self, *args, **kwargs): File "e:\text-generation-webui\installer_files\env\lib\site-packages\torch_init_.py", line 1705, in fn return compile(model, File "e:\text-generation-webui\installer_files\env\lib\site-packages\torch_init_.py", line 1723, in compile return torch._dynamo.optimize(backend=backend, nopython=fullgraph, dynamic=dynamic, disable=disable)(model) File "e:\text-generation-webui\installer_files\env\lib\site-packages\torch_dynamo\eval_frame.py", line 583, in optimize check_if_dynamo_supported() File "e:\text-generation-webui\installer_files\env\lib\site-packages\torch_dynamo\eval_frame.py", line 535, in check_if_dynamo_supported raise RuntimeError("Windows not yet supported for torch.compile") RuntimeError: Windows not yet supported for torch.compile

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

always

Screenshot

No response

Logs

22:52:36-341318 INFO     Loading mobiuslabsgmbh_Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-2bit-HQQ
22:52:36-341318 ERROR    Failed to load the model.
Traceback (most recent call last):
  File "e:\text-generation-webui\modules\ui_model_menu.py", line 213, in load_model_wrapper
    shared.model, shared.tokenizer = load_model(selected_model, loader)
  File "e:\text-generation-webui\modules\models.py", line 87, in load_model
    output = load_func_map[loader](model_name)
  File "e:\text-generation-webui\modules\models.py", line 393, in HQQ_loader
    from hqq.core.quantize import HQQBackend, HQQLinear
  File "e:\text-generation-webui\installer_files\env\lib\site-packages\hqq\core\quantize.py", line 238, in <module>
    class HQQLinear(torch.nn.Module):
  File "e:\text-generation-webui\installer_files\env\lib\site-packages\hqq\core\quantize.py", line 336, in HQQLinear
    def matmul_compile(self, *args, **kwargs):
  File "e:\text-generation-webui\installer_files\env\lib\site-packages\torch\__init__.py", line 1705, in fn
    return compile(model,
  File "e:\text-generation-webui\installer_files\env\lib\site-packages\torch\__init__.py", line 1723, in compile
    return torch._dynamo.optimize(backend=backend, nopython=fullgraph, dynamic=dynamic, disable=disable)(model)
  File "e:\text-generation-webui\installer_files\env\lib\site-packages\torch\_dynamo\eval_frame.py", line 583, in optimize
    check_if_dynamo_supported()
  File "e:\text-generation-webui\installer_files\env\lib\site-packages\torch\_dynamo\eval_frame.py", line 535, in check_if_dynamo_supported
    raise RuntimeError("Windows not yet supported for torch.compile")
RuntimeError: Windows not yet supported for torch.compile

System Info

win10
rtx 4060