lmdeploy icon indicating copy to clipboard operation
lmdeploy copied to clipboard
Published 1 month ago • verified publisher icon InternLM

[Feature] Please support Llama3.2 and Qwen2.5

Open mihara-bot opened this issue 1 year ago • 5 comments

Motivation

https://huggingface.co/collections/meta-llama/llama-32-66f448ffc8c32f949b04c8cf https://huggingface.co/collections/Qwen/qwen25-66e81a666513e518adb90d9e

Related resources

No response

Additional context

No response

mihara-bot avatar Sep 27 '24 06:09 mihara-bot

就模型结构而言，qwen2.5==qwen2，我已经跑起来了

the-nine-nation avatar Sep 29 '24 07:09 the-nine-nation

就模型结构而言，qwen2.5==qwen2，我已经跑起来了

Hi，但是我在用TurboMind Qwen 2.5会报错，提示不支持

mihara-bot avatar Sep 29 '24 07:09 mihara-bot

pytorch是可以的，turbomind或许需要适配，你可以尝试直接改代码底层，看能不能让引擎将qwen2.5认成qwen2

the-nine-nation avatar Sep 29 '24 09:09 the-nine-nation

pytorch是可以的，turbomind或许需要适配，你可以尝试直接改代码底层，看能不能让引擎将qwen2.5认成qwen2

谢谢您的建议！在官方正式支持前，也只能这么干了

mihara-bot avatar Sep 29 '24 09:09 mihara-bot

@mihara-bot turbomind是支持qwen2.5的，遇到了怎样的错误呢？

lvhan028 avatar Oct 09 '24 07:10 lvhan028