Andrew Lapp

@maxtheman I couldn't reproduce the error in your script. Is it possible https://github.com/outlines-dev/outlines/commit/aacc63392971fad29bbd810023fbe635ef7df328 fixed it? Please let me know if the issue still occurs on the latest version of outlines.

@m0g1cian opened an upstream issue: https://github.com/numba/numba/issues/9542 Per the thread, it appears to be an upstream bug on the numba side due to `UnicodeCharSeq` having trouble handling leading null byte `\x00`....

We might consider returning a `GenerationResult` object. e.g. ``` >>> result = outlines.generate.choice(model, ["Positive", "Negative"], sampler=BeamSampler(2))` >>> result.text "Positive" >>> result.relative_probs # probability relative to actual generations {"Positive": 0.6, "Negative":...

With beam search, you are guaranteed to explore all legal paths given that the number of legal paths is equal to the number of beams. This is why I suggest...

> You can walk the FSM created when calling `RegexFSM`, get all the possible token combinations, run them in one batch through the model and sum the path probabilities. I'm...

>I'm not proficient enough to tell if this can be implemented in outlines as well. No, it currently isn't. `cappr`'s implementation makes sense. It doesn't generate, it simply calculates the...

@br3no are there any new developments for fast-forward / accelerate?

@hugocool To have stable CFG-based JSON generation we need - 1) Tooling like https://github.com/dottxt-ai/outlines/blob/main/outlines/fsm/json_schema.py which, instead of converting to regex, constructs an appropriate Lark grammar. - 2) Completion of https://github.com/dottxt-ai/outlines/issues/1151...

@brandonwillard could you please ensure https://github.com/outlines-dev/outlines/pull/886 resolves the issue on your end or provide reproduction steps?

Yep, the PR disables caching on everything except `tests.test_cache` and `tests.benchmark` (benchmarks have their own cache handling)