mistral.rs
mistral.rs copied to clipboard
Support MLA for DeepSeek models
Refs:
- https://github.com/deepseek-ai/DeepSeek-V3/blob/4cc6253d5c225e2c5fea32c54573449c1c46470a/inference/model.py#L443
- https://github.com/sgl-project/sglang/pull/905/files#diff-5b9e34dd492bd8a14702a18b594721091092276fad1cf8736fba6ef1f33c1b04
- https://github.com/InternLM/lmdeploy/pull/1621/files#diff-daef4154c2a77eba9f2e444df958cc19b318ce248c09995080b344b174522dc5
Code Metrics Report
=============================================================================== Language Files Lines Code Comments Blanks =============================================================================== C Header 3 62 53 0 9 Dockerfile 1 41 22 10 9 JSON 11 104 103 0 1 Makefile 1 6 5 0 1 Python 80 3457 2984 98 375 Shell 1 63 26 18 19 Plain Text 3 3723 0 2413 1310 TOML 19 551 506 6 39 YAML 2 21 19 2 0 ------------------------------------------------------------------------------- Jupyter Notebooks 3 0 0 0 0 |- Markdown 2 77 32 31 14 |- Python 2 205 178 1 26 (Total) 282 210 32 40 ------------------------------------------------------------------------------- Markdown 53 4787 0 3646 1141 |- BASH 8 105 102 0 3 |- JSON 1 12 12 0 0 |- Python 7 121 109 0 12 |- Rust 20 695 587 0 108 |- TOML 2 75 63 0 12 (Total) 5795 873 3646 1276 ------------------------------------------------------------------------------- Rust 358 119660 107109 2336 10215 |- Markdown 161 1813 25 1646 142 (Total) 121473 107134 3982 10357 =============================================================================== Total 535 132475 110827 8529 13119 ===============================================================================