JSON Former Example: "No valid option generated in #select!"

[lolz0r]

The bug Guidance throws an error when trying to run the JSONformer example while using the 'mosaicml/mpt-7b-instruct' model. The error message asks to create a github issue as "this should not happen :)"

To Reproduce

import guidance
guidance.llm = guidance.llms.transformers.MPTChat('mosaicml/mpt-7b-instruct', device="cpu")

# define a jsonformer schema
json_schema = {
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "age": {"type": "number"},
        "is_student": {"type": "boolean"},
        "courses": {
            "type": "array",
            "items": {"type": "string"}
        }
    }
}

# run the guidance program
prompt = "Generate a person's information based on the following schema:"
program = guidance(prompt + "\n" + jsonformer2guidance(json_schema))
out = program()

System info (please complete the following information): MacOS 11.7.3, CPU only.

[slundberg]

Thanks can you include the guidance version you are using? 0.0.49 includes a fix for errors like this

[lolz0r]

This is my pip show

Name: guidance
Version: 0.0.49
Summary: A guidance language for controlling large language models.
Home-page: https://github.com/slundberg/guidance
Author: Scott Lundberg and Marco Tulio Ribeiro
Author-email: [email protected]
License: 
Location: /usr/local/lib/python3.9/site-packages
Requires: diskcache, nest-asyncio, numpy, openai, parsimonious, platformdirs, pygtrie, requests, tiktoken
Required-by: 

[Mihaiii]

I can confirm the issue with version 0.0.49.

@slundberg I replaced in the above script guidance.llm = guidance.llms.transformers.MPTChat('mosaicml/mpt-7b-instruct') with guidance.llm = guidance.llms.OpenAI("text-ada-001") and I get "The OpenAI API does not support Guidance pattern controls! Please either switch to an endpoint that does, or don't use the pattern argument to gen.".

Is this error expected? I'm asking because the very first example in readme uses OpenAi API with "gen" and it works just fine. The error message is strange.

[slundberg]

I can confirm the issue with version 0.0.49.

@slundberg I replaced in the above script guidance.llm = guidance.llms.transformers.MPTChat('mosaicml/mpt-7b-instruct') with guidance.llm = guidance.llms.OpenAI("text-ada-001") and I get "The OpenAI API does not support Guidance pattern controls! Please either switch to an endpoint that does, or don't use the pattern argument to gen.".

Is this error expected? I'm asking because the very first example in readme uses OpenAi API with "gen" and it works just fine. The error message is strange.

It is because that notebook uses the pattern=.... argument. We could add an option to remove that so OpenAI can work as well. It might also be nice to add examples of other formats like YAML etc.

[slundberg]

I am closing this issue for now because I don't get the error anymore with 0.0.55. Let me know if there is a problem still.