Breno Faria
Breno Faria
@njhill > Is it possible / relatively cheap to clone a `Guide` that has been built based on particular parameters before it is used, so that each copy may be...
I’ll try to have a look in the coming days!
I have a few questions: > It is not supported from SamplingParamters Can you elaborate on why you think placing the guided decoding parameters in the `SamplingParams` is a good...
@rkooo567 thanks, let me see if I understand it: The idea is that the logits processors will be asked to `prepare` their masks asynchronously and in the meantime the model...
@mmoskal thanks for your answer! I also would like to support ff-tokens since I think this would contribute to alleviate the performance issues. @njhill I’m not familiar with lm-format-enforcer, but...
I'm glad to see this effort in adding output schemas to the tool call spec! The main issue of not having a schema is that the server tool implementation is...
Just checking: has this been backported to 2.x? It would be very unfortunate if 2.19.2 does not contain this. I can't find any reference to it, though.