[Question] what happened if i only do create_index() on collection creation not everytime i insert to collection?

[utopictown]

was wondering because usually in SQL we only create index once for table but in example quick deploy we create index everytime we insert data to collection

[shiyu22]

@utopictown Hi, yes, you can call create_index() only once and Milvus will automatically build an index when the data size reaches the sealed segment size, which is determined by datacoord.segment.maxSize and datacoord.segment.sealProportion.

The difference is that if you manually call create_index after inserting data, Milvus will create an index for the remaining data. At this time, the amount of data does not need to reach the size of the sealed segment, but it is required to be higher than rootCoord.minSegmentSizeToEnableIndex.

I think all the data inserted into the system needs to be indexed, so create_index will be called after each data insertion. Of course, you can also use create Index once after creating the Milvus collection.

[utopictown]

@shiyu22 thank you for the answer, i have problem in querying the vectors, it took too long and never succeed, does this has anyhting to do with the indexing?

[shiyu22]

I think this is related to your data scale and hardware. You can run docker logs milvus-standalone to view the logs. If there are any error messages, you can submit an issue to the Milvus project.

BTW, you can first run Milvus on a small scale to check it.