lorax
lorax copied to clipboard
fix: load tokenizer/config with trust_remote_code
What does this PR do?
loading the tokenizer with remote code and unregistered class, will require user interactive input for confirming yes/no. That will break the normal processing.
Fixes #466
Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Was this discussed/approved via a Github issue or the discord / slack channel? Please add a link to it if that's the case.
- [ ] Did you write any new necessary tests?
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.