dottxt-ai
Add optional tokenizer paramers to generation
The title and the changes speak for themselves, I hope. In my case, this change was necessary to handle a bug with the Gemma 3 models whose fix involves passing additional parameters to the tokenizer when encoding the prompt. However, I think my changes allow for overall greater flexibility.
I didn't run the tests locally because it's a minimal change and I couldn't be bothered to install all of the additional libraries. I did run the pre-commit though and got no errors.
Hi @jkuntzer, thanks for opening a PR! We are close to releasing the v1 of Outlines that will modify the interface of models. Would you mind testing out the models that cause this issue with the v1 branch please?
Hi @RobinPicard, sure I can give it a try. However, the bug is not related to outlines; it's an inherent model problem in the transformers library, I believe. The problem was that I couldn't implement the fix in outlines.