Windows运行报错

[interfas24]

Windows编译安装正常， 运行

python -m ktransformers.local_chat --model_path deepseek-ai/DeepSeek-V2-Lite-Chat --gguf_path ./DeepSeek-V2-Lite-Chat-GGUF

聊天时报错

Chat: why sky is blue
Traceback (most recent call last):
  File "C:\Users\jqhan\Code\ktransformers\ktransformers\util\utils.py", line 159, in prefill_and_generate
    model._get_logits_warper(generation_config,device=inputs.device)
    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Code\Libs\Py\3.11\Lib\site-packages\torch\nn\modules\module.py", line 1928, in __getattr__
    raise AttributeError(
AttributeError: 'DeepseekV2ForCausalLM' object has no attribute '_get_logits_warper'. Did you mean: '_get_logits_processor'?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\jqhan\Code\ktransformers\ktransformers\local_chat.py", line 179, in <module>
    fire.Fire(local_chat)
  File "D:\Code\Libs\Py\3.11\Lib\site-packages\fire\core.py", line 135, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Code\Libs\Py\3.11\Lib\site-packages\fire\core.py", line 468, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
                                ^^^^^^^^^^^^^^^^^^^^
  File "D:\Code\Libs\Py\3.11\Lib\site-packages\fire\core.py", line 684, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jqhan\Code\ktransformers\ktransformers\local_chat.py", line 173, in local_chat
    generated = prefill_and_generate(
                ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jqhan\Code\ktransformers\ktransformers\util\utils.py", line 163, in prefill_and_generate
    model._get_logits_warper(generation_config)
    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Code\Libs\Py\3.11\Lib\site-packages\torch\nn\modules\module.py", line 1928, in __getattr__
    raise AttributeError(
AttributeError: 'DeepseekV2ForCausalLM' object has no attribute '_get_logits_warper'. Did you mean: '_get_logits_processor'?

[adonishong]

same issue here ...

[adonishong]

pull the latest code, reinstall by ./install.sh, and re-run, this issue have been fixed already

[CJJ1008]

Chat: who are you? Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/root/autodl-tmp/ktransformers/ktransformers/local_chat.py", line 197, in fire.Fire(local_chat) File "/root/miniconda3/lib/python3.12/site-packages/fire/core.py", line 135, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/lib/python3.12/site-packages/fire/core.py", line 468, in _Fire component, remaining_args = _CallAndUpdateTrace( ^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/lib/python3.12/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^ File "/root/autodl-tmp/ktransformers/ktransformers/local_chat.py", line 186, in local_chat generated = prefill_and_generate( ^^^^^^^^^^^^^^^^^^^^^ File "/root/autodl-tmp/ktransformers/ktransformers/util/utils.py", line 314, in prefill_and_generate generation_config, model_kwargs = model._prepare_generation_config( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1962, in getattr raise AttributeError( AttributeError: 'DeepseekV2ForCausalLM' object has no attribute '_prepare_generation_config'

I use Linux, and my issue is similar to yours, but even after reinstalling by bash install.sh, and rerunning, the problem still isn’t resolved.

[LouieBHLu]

same issue here, with linux

[UserNameForGithub03]

same issue with linux.

[zhizi42]

similar issue, when use qwen2 AttributeError: 'Qwen2MoeForCausalLM' object has no attribute '_prepare_generation_config' with wsl