Document more clearly ChatGPT partial support

[gkossakowski]

When experimenting in Playground with COT, I got this error:

  File "/home/pyodide/lmql/runtime/bopenai/openai_api.py", line 176, in chat_api
    assert "logit_bias" not in kwargs.keys(), f"Chat API models do not support advanced constraining of the output, please use no or less complicated constraints."
AssertionError: Chat API models do not support advanced constraining of the output, please use no or less complicated constraints.

So even fairly basic example (from a user PoV) is not supported by ChatGPT integration. I didn't find any mention of that in the doc.

See also https://github.com/eth-sri/lmql/issues/23

[lbeurerkellner]

Thanks for raising this. As explained in PR #25, there are some issue on the OpenAI end, which are hard for us to solve, although we are weighing off several options right now.

I will make sure to add some more details to https://docs.lmql.ai/en/stable/language/openai.html#openai-api-limitations, which documents this limitation wrt. Chat models.

[gkossakowski]

Yeah, I understand that this is ultimately a limitation of OpenAI's API. My gripe is only with the docs not making that limitation of LQML <> ChatGPT more prominent and understandable. This would help to determine whether LQML is a good fit for a use case at this moment.

PS. As a former PL nerd, let me say: great project!

[younes-io]

Yeah, I got the same issue BTW, the limitations link seems broken. The new version I suppose : https://lmql.ai/docs/models/openai.html#openai-api-limitations