Clarification on Code Implementation in DeepSeek vs Llama

[dog14230pp]

Dear Authors,

Thank you for providing such excellent work for the community to use!

I have a question regarding an implementation detail. In Line 338, it appears that the code is adapted from Llama. However, when looking closer, the implementation in DeepSeek seems to differ, particularly from Line 363 to Line 367, compared to Llama’s implementation in Line 223.

Could you explain the reasoning behind this difference? Were there specific considerations that led to this change?

I look forward to your response. Thank you again for your great work!

Best regards,

[kelsthevibe]

I am interested as well.

[rasrenato]

Hi! I tried to analyze the differences between DeepSeek-Coder-V2 and Llama, as mentioned in this issue. However, I couldn't find the source code in this repository—only licenses and the paper. Is the implementation of DeepSeek-Coder-V2 publicly available? If so, where can we find it?

Thanks in advance for your clarification! 🚀