Walter Nelson

Yes, I agree. I am running into the same issue now. This is particularly frustrating because of the arcane versioning of CUDA-related toolsets (i.e. the Python packages vs. CUDA vs....

I second the first question -- it would be useful to understand, because I very often see counterintuitive behaviour between unconstrained generation through raw `transformers` and constrained generation through `guidance`....

> Funny enough, for my use case, I actually prefer this behaviour, but I'm having trouble reliably getting it with my model/prompt, so some insight as to the internals of...

Thanks for that insight! I agree, I think greedy decoding is a very sensible and intuitive default (although sadly does not seem to quite hold for all models as of...