Add a file to Knowledge and display : "1 validation error for KnowledgeConfig......"

[fyxkszz]

Self Checks

• [x] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [x] I have searched for existing issues search for existing issues, including closed ones.
• [x] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [x] Please do not modify this template :) and fill in all the required fields.

Dify version

1.1.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

I created a new Knowledge and uploaded some excle files. These files were processed successfully. There was an error when adding files later.

Knowledge Using Hybrid Search Method

Here is my operation process

✔️ Expected Behavior

❌ Actual Behavior

[dosubot[bot]]

Hey @fyxkszz! 👋 I'm here to help you with bug fixes, answer questions, and assist you while waiting for a human maintainer. Let's squash those bugs together!

To resolve the Pydantic validation error indicating a missing required field weight_type in the KnowledgeConfig model, you need to ensure that the weights field in the RetrievalModel is properly initialized with a WeightModel that includes the weight_type attribute. Here's an example of how you can define the KnowledgeConfig with the required weight_type:

from pydantic import BaseModel
from typing import Literal, Optional

class WeightModel(BaseModel):
    weight_type: str
    vector_setting: Optional[WeightVectorSetting] = None
    keyword_setting: Optional[WeightKeywordSetting] = None

class RetrievalModel(BaseModel):
    search_method: Literal["hybrid_search", "semantic_search", "full_text_search"]
    reranking_enable: bool
    reranking_model: Optional[RerankingModel] = None
    reranking_mode: Optional[str] = None
    top_k: int
    score_threshold_enabled: bool
    score_threshold: Optional[float] = None
    weights: Optional[WeightModel] = None

class KnowledgeConfig(BaseModel):
    original_document_id: Optional[str] = None
    duplicate: bool = True
    indexing_technique: Literal["high_quality", "economy"]
    data_source: Optional[DataSource] = None
    process_rule: Optional[ProcessRule] = None
    retrieval_model: Optional[RetrievalModel] = None
    doc_form: str = "text_model"
    doc_language: str = "English"
    embedding_model: Optional[str] = None
    embedding_model_provider: Optional[str] = None
    name: Optional[str] = None

# Example initialization
knowledge_config = KnowledgeConfig(
    retrieval_model=RetrievalModel(
        search_method="semantic_search",
        reranking_enable=True,
        top_k=10,
        score_threshold_enabled=False,
        weights=WeightModel(
            weight_type="your_weight_type_here"
        )
    )
)

Make sure to replace "your_weight_type_here" with the appropriate value for weight_type ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[powertoredstar]

I also encountered this problem,too

[ensleep]

Any way to resolve this before upgrade?

[icyflower]

Similar operations, similar problems

[jqsl2012]

[indradevelop]

any update on this, same with version 1.1.2

[polischool]

Same problem on cloud hosted version.

[minamikik]

Same problems with cloud hosted ver.

[WTW0313]

We are actively working on resolving this issue. Thank you so much for your patience and understanding.

[Byzeal]

I also encountered this problem (V1.1.2) and would like to know how to solve it specifically??

[woaiso]

+1

[WTW0313]

A new version will be released soon to address this issue, so you won't have to wait for too long. Thank you all for your patience and understanding.

[SeesawLiu]

@WTW0313 the version 1.1.3 did't solve this problem,when you "save and process" a document and the database use rerank model,the weight_type is null,the backend service report "1 validation error for KnowledgeConfig\nretrieval_model.weights.weight_type\n Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]"

[WTW0313]

@WTW0313 the version 1.1.3 did't solve this problem,when you "save and process" a document and the database use rerank model,the weight_type is null,the backend service report "1 validation error for KnowledgeConfig\nretrieval_model.weights.weight_type\n Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]"

I can't reproduce this issue in version 1.1.3. Could you please provide the steps to reproduce the issue? If possible, screenshots or videos would be incredibly helpful.

[icyflower]

Add a file to old Knowledge, sometimes

[SeesawLiu]

like @icyflower said,the old knowledage database config did't have the parameter,when i try save config again to overwrite it,some success and some fail.some kdb update to "customized",and some update to null.

[1609676823]

When using dify version 1.1.3, newly created knowledge bases can upload files normally, but historically created knowledge bases still report an error:

1 validation error for KnowledgeConfig retrieval_model.weights.weight_type 
Input should be a valid string [type=string_type, input_value=None, input_type=NoneType] 
For further information visit https://errors.pydantic.dev/2.9/v/string_type

[owenouy]

I'm currently using version 1.1.2 and encountering the same issue. It only works correctly when adding all the required files to the knowledge base the first time. If I try to add more files later, it throws an error—the same one you're experiencing. It seems version 1.1.3 hasn't fixed this issue either!

[WTW0313]

I'm currently using version 1.1.2 and encountering the same issue. It only works correctly when adding all the required files to the knowledge base the first time. If I try to add more files later, it throws an error—the same one you're experiencing. It seems version 1.1.3 hasn't fixed this issue either!

Sorry about that. Currently, to avoid this error in old datasets, you can create a new dataset and upload the files again. We are working on a better solution for this issue, which will be compatible with old data.

[owenouy]

I'm currently using version 1.1.2 and encountering the same issue. It only works correctly when adding all the required files to the knowledge base the first time. If I try to add more files later, it throws an error—the same one you're experiencing. It seems version 1.1.3 hasn't fixed this issue either!

Sorry about that. Currently, to avoid this error in old datasets, you can create a new dataset and upload the files again. We are working on a better solution for this issue, which will be compatible with old data.

So, is the current workaround to create a new knowledge base with the additional documents, and then select both knowledge bases for searching?

[WTW0313]

You can give it a try. If the retrieval works well, it could certainly be a more efficient solution compared to simply duplicating all files into a new dataset.

[abbottdev]

Also having this issue after an upgrade to v1.1.3