Andrew Wang
Andrew Wang
Same issue
This is probably a side effect of interregular, which has implicit anchoring: https://github.com/MegaIng/interegular/issues/10
> btw @maxdebayser this is the chat template I have built. It's not an understatement to say I have spent 4+ hours simplifying it from the original, tuning the system...
Same on the Nemotron models.
They're whitespace tokens (\n, \t, \d, etc) which are typically allowed. You can disable them using the `CharacterLeveParserConfig`.
@rtqichen Bumping this. I am also running into similar issues with torch export, enabling this would be very useful
> if the tool use token is called, pull out the tool call JSON from the rest of the response (should be array) Given that guided decoding is not enabled...
I also wanted to ask a question about how `tool_call_ids` are going to be handled. I know HF says: > Note that we generate a random tool_call_id here. These IDs...
> @aw632 thats a great question for Hermes V2 we plan to pass back a dict for tool response as follows: > > {"tool_call_id": , "name": , "content": } >...
Btw, it looks like the stop reason/finish reason isn't accurately displaying "tool_calls" as the OpenAI API expects (this happens with named tool use too). I can open an issue for...