FastChat icon indicating copy to clipboard operation
FastChat copied to clipboard

Published 20 hours ago • verified publisher icon lm-sys

Specify ASCEND NPU for inference.

Open sunyi0505 opened this issue 11 months ago • 1 comments
trafficstars

Why are these changes needed?

When deploying inference services with ASCEND NPU, it is not possible to specify the card to be used. @infwinston @CodingWithTim

Related issue number (if applicable)

Checks

  • [x] I've run format.sh to lint the changes in this PR.
  • [x] I've included any doc changes needed.
  • [x] I've made sure the relevant tests are passing (if applicable).

sunyi0505 avatar Nov 29 '24 03:11 sunyi0505