transformers icon indicating copy to clipboard operation
transformers copied to clipboard

Pass device in Logits Processor's init

Open zucchini-nlp opened this issue 1 year ago • 6 comments

What does this PR do?

This PR adds the ability to pass in device when initializing LogitsProcessors and is one more step towards compile compatibility.

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

@gante

zucchini-nlp avatar Mar 22 '24 09:03 zucchini-nlp

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

  1. Cool, I did not notice that 2 and 3. Okay, thought we need it for consistency like we had with other new args in public classes. Will remove it and rebase later

zucchini-nlp avatar Mar 25 '24 15:03 zucchini-nlp

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

github-actions[bot] avatar Apr 22 '24 08:04 github-actions[bot]

Not stale

zucchini-nlp avatar Apr 22 '24 08:04 zucchini-nlp

This PR now can be reviewed. Rebased main and updated the changes. All the tests from RUN_SLOW=1 pytest tests/generation are passing on my end

zucchini-nlp avatar May 09 '24 20:05 zucchini-nlp