guidance icon indicating copy to clipboard operation
guidance copied to clipboard

Allow the LLM to dictate key order when generating JSON

Open wjn0 opened this issue 8 months ago • 2 comments

Is your feature request related to a problem? Please describe. _gen_json_object dictates key order.

Describe the solution you'd like Instead, allow the LLM to dictate it, reducing perplexity when the LLM has knowledge of the schema.

Describe alternatives you've considered As-is is probably fine for a lot of use cases, but for mine (similar to the public schema in #889) even off the shelf Llama 3 has knowledge of the specification likely due to it being present in its pretraining set so specifying key order may increase perplexity.

wjn0 avatar Jun 06 '24 15:06 wjn0