Suggestion prompt to return UnknownText

[bonadio]

Hi, first I would like to say that this library is fantastic.

I am working on a chatbot and I want to extract the users order, below is the sample prompt I am using.

When the user says the "correct" order or something "completely off" I get a good result, but sometimes I get incorrect results

Here is the prompt:

Given the following passage:

"do you have pizza "

Respond in JSON that satisfies the following type.

// Pizza order
export type PizzaOrder = (PizzaItem | UnknownText)[];

// Use this type for order items that match nothing else
export interface UnknownText {
    type: "unknown",
    text: string; // The text that wasn't understood
}

// Pizza item with quantity, toppings and observations
export type PizzaItem = {
    quantity: number;
    toppings: PizzaTopping[];
    observation?: string; 
};

// Pizza topping  
export type PizzaTopping = "Pepperoni" | "Mushrooms" | "Black Olives" | "unknown";

If I say "I would like two pepperoni pizzas" I get the correct result

[
    {"quantity": 2,
     "toppings": ["Pepperoni"]
     }
]

And when I say "is the sky blue?" I get the correct result too "unknown"

[{
"type": "unknown",
"text": "is the sky blue"
}]

But if I say something like "Do you have pizza" I get this wrong

[{
"quantity": 1,
"toppings": ["Pepperoni"]
}]

How could I improve this prompt to get the "unknown" in this case? I am using PALM2 text-bison-001, but this also happens in Gpt-3.5-turbo-instruct too.

Thanks

[DanielRosenwasser]

I think the answer I would give to this is that the "unknown" pattern is not a catch-all. In some cases, you can augment it by adding comments to describe what you want to happen. In other cases, you may be better-served by anticipating more classes of "bad input" by adding a feature. So in the case of "do you have pizza", you might try to interleave a different kind of response than just a Cart, or have a Response with both a cart and an answer property.

But otherwise, I think you will always have to give users an "out", double check with your users to say "is this what you wanted?"