The parameter 'n' in ChatOpenAI only creates one completion

[Tingmi88]

System Info

Langchain version: 0.0.190

Who can help?

No response

Information

• [X] The official example notebooks/scripts
• [ ] My own modified scripts

Related Components

• [X] LLMs/Chat Models
• [ ] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [ ] Output Parsers
• [ ] Document Loaders
• [ ] Vector Stores / Retrievers
• [ ] Memory
• [ ] Agents / Agent Executors
• [ ] Tools / Toolkits
• [ ] Chains
• [ ] Callbacks/Tracing
• [ ] Async

Reproduction

Initialize chat and message and specify n = 2

chat = ChatOpenAI(model="gpt-3.5-turbo", n=2) prompt_template = ChatPromptTemplate.from_template("Write a 10 words poem.") message = prompt_template.format_messages()

Get the response

response = chat(message)

Only one completion is obtained even though n specified as 2.

Expected behavior

The number of response contents should align with the n we specified in ChatOpenAI instantiation.

[devstein]

Generated by a 🤖

Answer

To get multiple completions, you should use the generate method instead of directly calling the chat object with a message. Replace the following lines:

response = chat(message)

with:

response = chat.generate([[message]])

This will give you the desired number of completions specified by the n parameter

Source

• tests/integration_tests/chat_models/test_openai.py

---

This response is meant to be helpful and save users time. It is not meant to be a precise solution.

Please leave a 👍 if this is helpful and 👎 if it is irrelevant

[Tingmi88]

@devstein Thank you so much for your reply and reference. It solved my issue. btw, if n > 2, then we can use

responses = chat._generate([message])