[Bug]: 知识库构建索引出错

[andylzming]

Is there an existing issue for this?

• [X] I have searched the existing issues

Describe the bug

milvus 版本 2.3.3 ， 使用 docker 部署。

存储情况如下：

[root@server volumes]# du ./ --max-depth=1 -h
367M	./etcd
8.9M	./minio
888M	./milvus
1.3G	./

运行情况如下：

[root@gpu-server milvus]# docker-compose ps
NAME                IMAGE                                      COMMAND                   SERVICE      CREATED       STATUS                 PORTS
attu                zilliz/attu:v2.3.2                         "docker-entrypoint.s…"   attu         6 hours ago   Up 6 hours             0.0.0.0:8000->3000/tcp, :::8000->3000/tcp
milvus-etcd         quay.io/coreos/etcd:v3.5.5                 "etcd -advertise-cli…"   etcd         6 hours ago   Up 6 hours (healthy)   2379-2380/tcp
milvus-minio        minio/minio:RELEASE.2023-03-20T20-16-18Z   "/usr/bin/docker-ent…"   minio        6 hours ago   Up 6 hours (healthy)   0.0.0.0:9000-9001->9000-9001/tcp, :::9000-9001->9000-9001/tcp
milvus-standalone   milvusdb/milvus:v2.3.3                     "/tini -- milvus run…"   standalone   6 hours ago   Up 6 hours (healthy)   0.0.0.0:19530->19530/tcp, :::19530->19530/tcp

错误日志如下：

[2024-03-21 20:43:06,020: ERROR/MainProcess] RPC error: [insert_rows], <DataNotMatchException: (code=1, message=Attempt to insert an unexpected field to collection without enabling dynamic field)>, <Time:{'RPC start': '2024-03-21 20:43:06.017347', 'RPC error': '2024-03-21 20:43:06.020646'}>
[2024-03-21 20:43:06,021: ERROR/MainProcess] Failed to insert batch starting at entity: 0/11
[2024-03-21 20:43:06,021: ERROR/MainProcess] Failed to insert batch starting at entity: 0/11
[2024-03-21 20:43:06,021: ERROR/MainProcess] consume document failed
Traceback (most recent call last):
  File "/app/api/core/indexing_runner.py", line 70, in run
    self._load(
  File "/app/api/core/indexing_runner.py", line 667, in _load
    index_processor.load(dataset, chunk_documents)
  File "/app/api/core/rag/index_processor/processor/paragraph_index_processor.py", line 59, in load
    vector.create(documents)
  File "/app/api/core/rag/datasource/vdb/vector_factory.py", line 121, in create
    self._vector_processor.create(
  File "/app/api/core/rag/datasource/vdb/milvus/milvus_vector.py", line 75, in create
    self.add_texts(texts, embeddings)
  File "/app/api/core/rag/datasource/vdb/milvus/milvus_vector.py", line 101, in add_texts
    raise e
  File "/app/api/core/rag/datasource/vdb/milvus/milvus_vector.py", line 95, in add_texts
    ids = self._client.insert(collection_name=self._collection_name, data=batch_insert_list)
  File "/usr/local/lib/python3.10/site-packages/pymilvus/milvus_client/milvus_client.py", line 206, in insert
    raise ex from ex
  File "/usr/local/lib/python3.10/site-packages/pymilvus/milvus_client/milvus_client.py", line 198, in insert
    res = conn.insert_rows(collection_name, insert_batch, timeout=timeout)
  File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 127, in handler
    raise e from e
  File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 123, in handler
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 162, in handler
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 102, in handler
    raise e from e
  File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 68, in handler
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/pymilvus/client/grpc_handler.py", line 501, in insert_rows
    request = self._prepare_row_insert_request(
  File "/usr/local/lib/python3.10/site-packages/pymilvus/client/grpc_handler.py", line 482, in _prepare_row_insert_request
    return Prepare.row_insert_param(
  File "/usr/local/lib/python3.10/site-packages/pymilvus/client/prepare.py", line 422, in row_insert_param
    return cls._parse_row_request(request, fields_info, enable_dynamic, entities)
  File "/usr/local/lib/python3.10/site-packages/pymilvus/client/prepare.py", line 370, in _parse_row_request
    raise DataNotMatchException(message=ExceptionsMessage.InsertUnexpectedField)
pymilvus.exceptions.DataNotMatchException: <DataNotMatchException: (code=1, message=Attempt to insert an unexpected field to collection without enabling dynamic field)>
[2024-03-21 20:43:06,028: INFO/MainProcess] Processed dataset: bd66a1d2-d871-42c4-8fe7-4275be32a591 latency: 6.201149852946401
[2024-03-21 20:43:06,064: INFO/MainProcess] Task tasks.document_indexing_task.document_indexing_task[0df93ed7-9cae-4670-842d-8032b3d5852b] succeeded in 6.23699377477169s: None

Expected Behavior

No response

Steps/Code To Reproduce behavior

No response

Environment details

- Hardware/Softward conditions (OS, CPU, GPU, Memory):
- Method of installation (Docker, or from source):
- Milvus version (v0.3.1, or v0.4.0):
- Milvus configuration (Settings you made in `server_config.yaml`):

Anything else?

No response

[zhuwenxing]

Attempt to insert an unexpected field to collection without enabling dynamic field

maybe you inserted some fields that were not defined in the schema. for example, your schema is id, doc_id, text, but you insert data with id, doc_id, text, name.

[andylzming]

Attempt to insert an unexpected field to collection without enabling dynamic field

maybe you inserted some fields that were not defined in the schema. for example, your schema is id, doc_id, text, but you insert data with id, doc_id, text, name.

Using dify platform knowledge base to connect milvus, I do not know how to implement it.