instructor
instructor copied to clipboard
Instructor multiclass classification is not providing proper result with gpt-3.5-turbo
- [ ] This is an actually a bug report.
- [x] I am not getting good LLM Results
- [ ] I have tried asking for help in the community on discord or discussions and have not received a response.
- [ ] I have tried searching the documentation and have not found an answer.
What Model are you using?
- [x] gpt-3.5-turbo
- [ ] gpt-4-turbo
- [ ] gpt-4
- [ ] Other (please specify)
Describe the bug Instructor text classification tutorial is not working. I was trying to run the multi-class classification code using this tutorial When I use gpt-3.5-turbo the output returns contain all labels instead of the right ones. Check the following code and output
client = instructor.patch(OpenAI())
def multi_classify(data: str) -> MultiClassPrediction:
"""Perform multi-label classification on the input text."""
return client.chat.completions.create(
model="gpt-3.5-turbo",
response_model=MultiClassPrediction,
messages=[
{
"role": "user",
"content": f"Classify the following ticket: {data}",
},
],
) # type: ignore
result = multi_classify("I didn't get bill.")
result.model_dump_json(indent=2)
# '{\n "class_labels": [\n "tech_issue",\n "billing",\n "general_query"\n ]\n}'
To Reproduce Try the instructor tutorial mentioned above which use gpt-3.5-turbo and see the results of multiclass classification.
Expected behavior Ideally this should be the response
# '{\n "class_labels": [\n "billing"\n ]\n}'
I'll work on the example a bit it has to do with prompting