baml icon indicating copy to clipboard operation
baml copied to clipboard

Update SAP Parser to infer a better end of string when the LLM drops multiple characters

Open tomsquest opened this issue 6 months ago • 7 comments

Hi,

I am starting to play with Baml, and I am very excited about it.

Problem

I stumble upon a strange case, where a test in PromptFilddle is failing, but the LLM output seems correct. Indeed, the test does not pass with the reason of a missing field, but the LLM seemed to have a correct output.

Detail

Here is the Raw llm Response as printed in PromptFiddle:

{
  "title": "L'année où j'ai vécu selon la Bible: Ou l'humble quête d'un homme qui cher"rating": 4,
  "volume": null,
  "description": null,
  "comment": null,
  "dateRead": null,
  "url": [
    "https://www.amazon.fr/Lann%C3%A9e-jai-v%C3%A9cu-selon-Bible/dp/2742789928/ref=sr_1_3?keywords=jacobs+bible"
  ],
  "slug": "lannee-ou-jai-vecu-selon-la-bible-ou-lhumbe-quete-dun-homme-qui-chercha-a-suivre-la-bible-aussi-litteralement-que-possible",
  "fiction": false
}

And the test fails with: Failed to coerce value: Error parsing '<root>': Missing required field: slug

Screenshot

image

Link to PromptFiddle

https://www.promptfiddle.com/extract-book-info-ystjv

tomsquest avatar Jul 31 '24 14:07 tomsquest