Andrew Lapp

Likely related to https://github.com/outlines-dev/outlines/issues/985 I'm working on a fix to a few json schema issues which have appeared. Thank you for your patience.

Likely related to https://github.com/outlines-dev/outlines/issues/609

To generate choices with `models.openai`, Outlines tokenizes the `choices` and sets the `logit_bias` (token filtering) API argument such that there is ~0% chance of any other token being selected. There...

I couldn't reproduce. Could you consider trying a tighter integration via https://github.com/outlines-dev/outlines/blob/main/docs/reference/models/llamacpp.md

This seems to be the same as the issue I ran into in https://github.com/outlines-dev/outlines/issues/796 I'll be working on getting CFG and the parser in a good state over the coming...

> So can this feature still be added? A chat template is really needed in many applications. Yes, if you'd like to introduce it in a PR, I'll review and...

Yes that's a risk but IMO it wouldn't substantially increase the maintenance requirements of the library: 1. Stateless sampling augmentation processors are quite a bit simpler than stateful structured generation...

@maximilianmordig can you please try again `main` since the following PR has merged? https://github.com/dottxt-ai/outlines/pull/1067

`outlines.processors` supports a number of array frameworks via `dlpack` copy-free type conversions. These are incredibly efficient and have near-zero overhead. It seems Jax supports dlpack https://jax.readthedocs.io/en/latest/jax.dlpack.html I'm glad you're interested...

Is this resolved by https://github.com/vllm-project/vllm/pull/7654