fabricator
fabricator copied to clipboard
Guided Generation for Syntax dependent tasks
Tools like Guidance help during text generation by not necessarily improving on the prompt.
A new paper called "Efficient Guided Generation for Large Language Models" does the same but with a cheaper runtime. One can provide a regex with which the model is guided during text generation. This might help with syntax-heavy tasks. E.g NER token lists ([0, 1, 2, 3]
), or some inline tagging Alex B-PER is O going O to O Los B-LOC Angeles I-LOC
.
The algorithm requires access to the generation task itself. This would therefore only work with self-hosted models.