text-generation-inference icon indicating copy to clipboard operation
text-generation-inference copied to clipboard

add intel xpu support for TGI

Open sywangyi opened this issue 1 year ago • 6 comments

What does this PR do?

Fixes # (issue)

Before submitting

  • [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • [ ] Did you read the contributor guideline, Pull Request section?
  • [ ] Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case.
  • [ ] Did you make sure to update the documentation with your changes? Here are the documentation guidelines, and here are tips on formatting docstrings.
  • [ ] Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

sywangyi avatar Jan 24 '24 01:01 sywangyi

Overall looks pretty good !

I think we'll move to propre enum SYSTEM instead of IS_XXXX since there's no way a user could be running 2 devices simultaneously. But we'll take care of that in a subsequent PR if that's OK.

Narsil avatar Jan 26 '24 13:01 Narsil

thanks very much @Narsil and @OlivierDehaene for your comment. adding @yao-matrix

sywangyi avatar Jan 29 '24 04:01 sywangyi

@Narsil @OlivierDehaene I add xpu smi in env runtime, do you think is it proper to add this? this is to dump intel XPU version.

sywangyi avatar Feb 01 '24 01:02 sywangyi

@Narsil @OlivierDehaene I add xpu smi in env runtime, do you think is it proper to add this? this is to dump intel XPU version.

It's OK in the current iteration (since showing N/A on most systems).

Narsil avatar Feb 01 '24 08:02 Narsil

Feel free to open up the draft whenever you're OK so we can run the tests + merge.

Narsil avatar Feb 01 '24 09:02 Narsil

@Narsil could you review the PR?done by my side.

sywangyi avatar Mar 21 '24 06:03 sywangyi

@Narsil @mfuntowicz CI node is ready, could we move on?

sywangyi avatar Apr 17 '24 03:04 sywangyi

@Narsil @mfuntowicz , pls feel free to let us know any issues, since CI is ready, I think we can launch CI and merge this PR, thx.

cc @kding1

yao-matrix avatar Apr 25 '24 00:04 yao-matrix