Andrew Lapp
Andrew Lapp
### What behavior of the library made you think about the improvement? In JSON Schema `safe_subset` mode - whitespace cannot continue infinitely https://github.com/outlines-dev/outlines/pull/916 - `integer` and `string` cannot continue infinitely...
# Overview The [repetition problem](https://arxiv.org/abs/2012.14660) of language models combined with patterns allowing for infinite-length fields results in broken JSON Schema outputs. This was addressed previously for infinite whitespaces issues by...
### What behavior of the library made you think about the improvement? Per [this PR comment](https://github.com/dottxt-ai/outlines/pull/1067#issuecomment-2251396332), the CFG implementation obviates the following: - `StopAtEosFSM` - `RegexFSM` Potentially no longer useful:...
Per https://github.com/dottxt-ai/outlines/pull/1067#issuecomment-2251396332 Here are some necessary performance and correctness improvements. ### Ensure parser allows ambiguous terminals The grammar `?start: /ab*/ /bc?/` with a generation of `abbbb` [doesn't allow a next...
### Describe the bug Since a few days ago the hub has been unable to render any tensorboards, instead displaying the following message > No dashboards are active for the...
Changes to make the code run on RTX 4090 / 3090. Fixes https://github.com/KellerJordan/modded-nanogpt/issues/29 [Runs in 2 hours 3 minutes](https://gist.github.com/lapp0/ff6f10c3cd6d0aefb28a49681a44b78c), Runs range from 3.275 to 3.285. This finished at 3.2817, These...
### Overview This PR implements GQA, which reduces KV projection size and attention overhead. GQA decreases sample efficiency, requiring ~8% more training steps, but improving per-step performance by 14%. Performance...
### What behavior of the library made you think about the improvement? Users may not be aware of the magnitude of different use cases. ### How would you like it...