[Bug] llama-factory训练报错：ModuleNotFoundError: No module named 'cpuinfer_ext.sft_moe'; 'cpuinfer_ext' is not a package

[slin000111]

检查清单

• [x] 1. 我已经搜索过相关问题，但未能获得预期的帮助
• [ ] 2. 该问题在最新版本中尚未修复
• [ ] 3. 请注意，如果您提交的BUG相关 issue 缺少对应环境信息和最小可复现示例，我们将难以复现和定位问题，降低获得反馈的可能性
• [ ] 4. 如果您提出的不是bug而是问题，请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭
• [ ] 5. 为方便社区交流，我将使用中文/英文或附上中文/英文翻译（如使用其他语言）。未附带翻译的非中文/英语内容可能会被关闭

问题描述

https://github.com/hiyouga/LLaMA-Factory/blob/main/examples/train_lora/deepseek2_lora_sft_kt.yaml 用这个脚本训练报错，如下：

Injecting model.layers.1.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.1.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.1.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.1.self_attn.kv_a_layernorm as default
Injecting model.layers.1.self_attn.kv_b_proj as default
Injecting model.layers.1.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.1.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Traceback (most recent call last):
  File "/mnt/workspace/miniconda3/envs/LF_KT/bin/llamafactory-cli", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/mnt/workspace/LLaMA-Factory/src/llamafactory/cli.py", line 24, in main
    launcher.launch()
  File "/mnt/workspace/LLaMA-Factory/src/llamafactory/launcher.py", line 157, in launch
    run_exp()
  File "/mnt/workspace/LLaMA-Factory/src/llamafactory/train/tuner.py", line 132, in run_exp
    _training_function(config={"args": args, "callbacks": callbacks})
  File "/mnt/workspace/LLaMA-Factory/src/llamafactory/train/tuner.py", line 91, in _training_function
    run_sft_kt(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)
  File "/mnt/workspace/LLaMA-Factory/src/llamafactory/train/ksft/workflow.py", line 47, in run_sft
    model = load_model(tokenizer, model_args, finetuning_args, training_args.do_train)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/LLaMA-Factory/src/llamafactory/model/loader.py", line 151, in load_model
    model = load_kt_pretrained_model(config, model_args)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/LLaMA-Factory/src/llamafactory/model/model_utils/ktransformers.py", line 110, in load_kt_pretrained_model
    optimize_and_load_gguf(model, optimize_config_path, gguf_path, config)
  File "/mnt/workspace/miniconda3/envs/LF_KT/lib/python3.11/site-packages/ktransformers/optimize/optimize.py", line 128, in optimize_and_load_gguf
    inject(module, optimize_config, model_config, weights_loader)
  File "/mnt/workspace/miniconda3/envs/LF_KT/lib/python3.11/site-packages/ktransformers/optimize/optimize.py", line 42, in inject
    inject(child, child_optimization_dict, model_config, gguf_loader, child_prefix)
  File "/mnt/workspace/miniconda3/envs/LF_KT/lib/python3.11/site-packages/ktransformers/optimize/optimize.py", line 42, in inject
    inject(child, child_optimization_dict, model_config, gguf_loader, child_prefix)
  File "/mnt/workspace/miniconda3/envs/LF_KT/lib/python3.11/site-packages/ktransformers/optimize/optimize.py", line 42, in inject
    inject(child, child_optimization_dict, model_config, gguf_loader, child_prefix)
  File "/mnt/workspace/miniconda3/envs/LF_KT/lib/python3.11/site-packages/ktransformers/optimize/optimize.py", line 31, in inject
    module_cls=getattr(__import__(import_module_name, fromlist=[""]), import_class_name)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/LF_KT/lib/python3.11/site-packages/ktransformers/operators/experts.py", line 32, in <module>
    from cpuinfer_ext.sft_moe import SFT_MOEConfig, SFT_MOE
ModuleNotFoundError: No module named 'cpuinfer_ext.sft_moe'; 'cpuinfer_ext' is not a package

复现步骤

训练命令：

llamafactory-cli train examples/train_lora/deepseek2_lora_sft_kt.yaml

环境信息

环境：

llamafactory                             0.9.4.dev0
ktransformers                            0.3.2+cu121torch27fancy

机器不支持AMX

[JimmyPeilinLi]

请问是使用的3.10版本的python嘛？如果是请换成>=3.11，我们后面会统一修正相关文档

[slin000111]

请问是使用的3.10版本的python嘛？如果是请换成>=3.11，我们后面会统一修正相关文档

感谢你的回复！python版本是3.11。我是通过源码安装的ktransformers，看起来是我的安装有问题。用官方编译的whl包安装后没这个报错了。