OpenAI API support - Langchain passes prompt as a list instead of str

[yanivshimoni89]

System Info

Ubuntu 22.04 A100 TGI docker version 1.4.3

Information

• [X] Docker
• [ ] The CLI directly

Tasks

• [X] An officially supported command
• [ ] My own modifications

Reproduction

Using langchain and create QA Retrieval Chain. The chain passes prompt as a list instead of str and the api should accept it.

Expected behavior

As mentioned in https://github.com/vllm-project/vllm/issues/186

[drbh]

Hi @yanivshimoni89 it's unclear if this is the intended functionality of the original openai chat api.

Looking into the vllm issue linked above it looks like theres only been a temporary fix where the first value of the list is treated as the prompt. (issue above refers to this PR https://github.com/vllm-project/vllm/pull/323#issuecomment-1615254198)

This fix is small and can be added to TGI but a longer term solution (update langchain to send a string) would be best

[yanivshimoni89]

Hi, @drbh I comprehend your viewpoint, and I concur. However, this functionality is currently not incorporated in TGI, which is a feature of the OpenAI API. As such, I believe it should be addressed in TGI. Regardless, I would appreciate being informed about any progress on this matter.