支持运行在昇腾环境的模型么

[suwief]

Search before asking

• [X] I had searched in the issues and found no similar feature requirement.

Description

本地环境 只有昇腾的卡，以及适配过的大模型

Use case

支持昇腾环境模型的部署推理

Related issues

No response

Feature Priority

None

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

[csunny]

目前应该还不支持，没有特殊适配过。

[zfanswer]

挺好奇你用哪个大模型，怎么做的升腾gpu的适配？

[suwief]

挺好奇你用哪个大模型，怎么做的升腾gpu的适配？

华为这边支持还挺多的，试了llama、通义千文

[suwief]

目前应该还不支持，没有特殊适配过。

是指多模型管理模块部分需要改代码么，还是说稍改一下脚本代码就能运行？

[github-actions[bot]]

This issue has been marked as stale, because it has been over 30 days without any activity.

[qidanrui]

目前应该还不支持，没有特殊适配过。

是指多模型管理模块部分需要改代码么，还是说稍改一下脚本代码就能运行？

我们现在假设模型基于cuda的驱动，如果模型能在昇腾卡上跑起来，应该是可以尝试一下的

[github-actions[bot]]

This issue has been marked as stale, because it has been over 30 days without any activity.

[github-actions[bot]]

This issue bas been closed, because it has been marked as stale and there has been no activity for over 7 days.