Andrew Lapp

@andupotorac @sharanry I don't see any documentation supporting structured generation. Even if a `logit_bias` argument was available, we still wouldn't be able to perform structured generation. In structured generation, the...

Thanks for the well documented issue! This appears to be an issue with our enum handling in `json_schema.py`, specifically calling `json.dumps` https://github.com/outlines-dev/outlines/blob/60e89f5706e3d0f9837e271e04a39fb6e81d92df/outlines/fsm/json_schema.py#L275 ``` >>> PersonInfo.model_json_schema() {'$defs': {'Emotion': {'enum': ['开心', '难过',...

Please provide comments for these issues. I will create the **New Issues** once they're refined / approved. # New Issues: Direct `CFGGuide` Fixes We should create a milestone for `CFGGuide`....

Thanks, I'll update `docs/reference/cfg.md` with a new section, and have the warning link there.

I have added rejection sampling. It checks each token for acceptance, starting with highest logprob, completing once **one** sample is accepted. This is effectively greedy sampling. This behavior is documented...

> As a I've signaled here in 28 April [#788 (comment)](https://github.com/outlines-dev/outlines/pull/788#issuecomment-2080167390). > > I've been working on a parser recently which I would say is around 95% of progress. It's...

Thanks so much, please let me know if you have any questions!

Great questions! Converting it to a filter is a bit hacky IMO, but may be the simplest solution and doesn't require an upstream change. Alternatively we could apply logits processing...

Rather than implementing a new logits processor, I'm awaiting correspondence with the ExLlamaV2 maintainer, turboderp, regarding whether a `logits_processor` argument would be acceptable within their sampler.

>So I thought some of the steps here overlap with our current logits processor. But yeah very much happy to get advice here since this is just making the exllamav2...