outlines icon indicating copy to clipboard operation
outlines copied to clipboard

Give the possibility to obtain the full response when calling the vLLM generate function

Open alonsosilvaallende opened this issue 4 months ago • 4 comments

I'm using InspectAI to evaluate language models. In particular, I'm evaluating the benefits of structured text generation using Outlines with language models. I would like to obtain the full response when calling the vLLM generate function since InspectAI expects to get the full response. Would it be possible to give the possibility to the user to get the full response. The default should still be the same as now which is a filtered response.

alonsosilvaallende avatar Oct 08 '24 10:10 alonsosilvaallende