lmql
lmql copied to clipboard
Allow distribution in the middle of a prompt
Since guidance is supporting it, could we add support for distributions not only at the end but also in the middle of the program?
Are you looking just for a VAR in [a,b,c,..]
constraint or are you interested in normalised probabilities across all possible values?
I am looking for normalized probabilities across the possible values. In guidance, this can be done by specifying the logprob
parameter in a select statement (see guidance - select).