question classifier are always classified into the first class

[ZxnSnowy]

Self Checks

• [x] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [x] I have searched for existing issues search for existing issues, including closed ones.
• [x] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
• [x] Please do not modify this template :) and fill in all the required fields.

Dify version

1.1.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

No matter what question is asked, it will be classified into category 1. It is normal to create category questions at first, but after a few previews, it will only be classified into category 1

I tried to change the order of the questions and put the same category into category 1：

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

[dosubot[bot]]

@ZxnSnowy Hello there! I'm here to help you with bugs, questions, and becoming a contributor. Let's tackle this issue together while we wait for a human maintainer.

I found some similar discussions that might be relevant to your issue:

1. Question Classification function in workflow maybe abnormal - This discussion is unsolved and was last updated on December 30, 2024 ^[1].

2. Help for use Question Classifier - This is another unsolved discussion, last updated on August 25, 2024 ^[2].

3. The classifier cannot understand the meaning - This unsolved discussion was last updated on June 18, 2024 ^[3].

Regarding your issue, a common cause for a question classifier to consistently classify inputs into the same category could be related to the configuration settings or the inference model used. You might want to check the following:

• Ensure that the classification labels/descriptions are distinct and well-defined to help the model differentiate between categories.
• Consider selecting a different inference model if the current one is not performing as expected.
• Review any advanced settings, such as additional instructions or memory features, that might affect classification accuracy ^[4][5].

Adjusting these settings might help improve the accuracy of the classification.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[Sakura4036]

Wrong Categories！The Question Classifier is not very smart，so Categories should be simple nouns rather than overly complex

[ZxnSnowy]

I tried qwen:7b and 14b, and found that the classification effect of 7b is much better than that of 14b, 14b will only classify the problems into the first category, why?

[Sakura4036]

yes， the classification performance of qwen 2.5:7b is better than 14b. I'm also confused

[Sakura4036]

When the LLM of the problem classifier does not return the correct format, the code will return the first class to prevent errors.

[leftchest]

The classification node memory of this problem has been invalid since version 1.0

[dosubot[bot]]

Hi, @ZxnSnowy. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary

• Bug reported in self-hosted Docker version 1.1.0: questions misclassified into the first category.
• I suggested checking configuration settings and inference models.
• Sakura4036 noted simpler category names might improve classification.
• Observations: 7b model performs better than 14b model, which misclassifies questions.
• Leftchest mentioned classification node memory has been invalid since version 1.0.

Next Steps

• Please confirm if this issue is still relevant to the latest version of Dify.
• If no further updates are provided, the issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!