Added Eagle training support for Kimi-K2
add support for Kimi-K2 eagle train
-
add target model for Kimi-K2 in specforge/modeling/target/kimi_k2.py
-
add Kimi-K2 config in configs/kimi-k2-eagle3.json
-
fix chat template in specforge/data/template.py
-
When generating the hidden layer, the special dialogue template of Kimi-K2 has been adapted in specforge/data/preprocessing.py
-
The tokenizer of the Kimi-K2 model cannot automatically use the fasttokenizer. A script is used to generate tokenizer.json, enabling it to use the fasttokenizer interface.
[!WARNING] Gemini is unable to generate a summary due to a potential policy violation.
Can you fix the conflict?
Can you fix the conflict?
Can you fix the conflict?
I have resolved the conflict based on upstream/main and re - submitted the code.
@xuhaojie-2025 Trying to use this for kimi-k2-0905 but having a bit of a time getting it working. Library issues, some stray bad lines, not using trust_remote_code in various places, outdated kimi_k2.py with bad refs to qk_head_dim, etc. I can struggle through but I'm wondering if perhaps you have an updated or functional branch/commit somewhere I can look at?