Example notebook of OpenAI got ConstraintException

[nullpointer0xffff]

The bug When using latest version of guidance '0.1.15' and run notebook multiple times ( < 10), it failed with error:

ConstraintException: The model attempted to generate b'THE CAPITAL OF GREENLAND IS NUUK.<|inval...' after the prompt `b'...enland?<|im_end|>\n<|im_start|>assistant\n'`, but that does
not match the given grammar constraints! Since your model is a remote API that does not support full guidance
integration we cannot force the model to follow the grammar, only flag an error when it fails to match.
You can try to address this by improving the prompt, making your grammar more flexible, rerunning with
a non-zero temperature, or using a model that supports full guidance grammar constraints.

This exception also got raised when using select, json etc. gen functionalities with OpenAI models.

To Reproduce Note: you might need to run multiple times before getting the ConstraintException.

• gen call:

from guidance import system, user, assistant

# this relies on the environment variable OPENAI_API_KEY being set
gpt35 = models.OpenAI('gpt-3.5-turbo')

lm = gpt35

with system():
    lm += "You only speak in ALL CAPS."

with user():
    lm += "What is the captial of Greenland?"

with assistant():
    lm += gen('answer', max_tokens=20)

• another example:

gpt = models.OpenAI("gpt-4")

with system():
    lm = gpt + "You are a cat expert."

with user():
    lm += "What are the smallest cats?"

with assistant():
    lm += gen("answer", temperature=0.001, max_tokens=15)

• select:

with system():
    lm = gpt + "Answer to the questions only with YES or NO."
with user():
    lm += """Life its Hard? """
with assistant():
    lm += "Answer = "

##USE SELECT OR GEN
with assistant():
    lm += select(['YES','NO'], 'answer')

System info (please complete the following information):

• OS (e.g. Ubuntu, Windows 11, Mac OS, etc.): Mac OS, jupyter notebook, this doesn't matter as it was also tested in other environment on my server (Linux).
• Guidance Version (guidance.__version__): 0.1.15
• OpenAI version: 1.30.4