mistral.rs icon indicating copy to clipboard operation
mistral.rs copied to clipboard

Support MLA for DeepSeek models

Open EricLBuehler opened this issue 10 months ago • 1 comments

Refs:

  • https://github.com/deepseek-ai/DeepSeek-V3/blob/4cc6253d5c225e2c5fea32c54573449c1c46470a/inference/model.py#L443
  • https://github.com/sgl-project/sglang/pull/905/files#diff-5b9e34dd492bd8a14702a18b594721091092276fad1cf8736fba6ef1f33c1b04
  • https://github.com/InternLM/lmdeploy/pull/1621/files#diff-daef4154c2a77eba9f2e444df958cc19b318ce248c09995080b344b174522dc5

EricLBuehler avatar Apr 19 '25 23:04 EricLBuehler

Code Metrics Report
  ===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 C Header                3           62           53            0            9
 Dockerfile              1           41           22           10            9
 JSON                   11          104          103            0            1
 Makefile                1            6            5            0            1
 Python                 80         3457         2984           98          375
 Shell                   1           63           26           18           19
 Plain Text              3         3723            0         2413         1310
 TOML                   19          551          506            6           39
 YAML                    2           21           19            2            0
-------------------------------------------------------------------------------
 Jupyter Notebooks       3            0            0            0            0
 |- Markdown             2           77           32           31           14
 |- Python               2          205          178            1           26
 (Total)                            282          210           32           40
-------------------------------------------------------------------------------
 Markdown               53         4787            0         3646         1141
 |- BASH                 8          105          102            0            3
 |- JSON                 1           12           12            0            0
 |- Python               7          121          109            0           12
 |- Rust                20          695          587            0          108
 |- TOML                 2           75           63            0           12
 (Total)                           5795          873         3646         1276
-------------------------------------------------------------------------------
 Rust                  358       119660       107109         2336        10215
 |- Markdown           161         1813           25         1646          142
 (Total)                         121473       107134         3982        10357
===============================================================================
 Total                 535       132475       110827         8529        13119
===============================================================================
  

github-actions[bot] avatar Apr 19 '25 23:04 github-actions[bot]