milvus icon indicating copy to clipboard operation
milvus copied to clipboard

[Bug]: Use the Go client to delete the data. After a while, use attu's query interface to query the data. Only part of it was deleted.

Open dushulin opened this issue 1 year ago • 16 comments
trafficstars

Is there an existing issue for this?

  • [X] I have searched the existing issues

Environment

- Milvus version:2.3.2
- Deployment mode(standalone or cluster):cluster
- MQ type(rocksmq, pulsar or kafka):    pulsar 
- SDK version(e.g. pymilvus v2.0.0rc2):go sdk v2.3.1
- OS(Ubuntu or CentOS): CentOS
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

some fileds: image go code(delete func) image after run this code, no a error info After a while, use attu's query interface to query the data. Only part of it was deleted. image

Expected Behavior

All entities have been deleted and cannot be queried

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response

dushulin avatar Apr 17 '24 08:04 dushulin

When the total amount of data to be deleted is more than 40, this problem will not occur. When the total amount of data is more than 200, this problem will occur.

dushulin avatar Apr 17 '24 08:04 dushulin

/assign @ThreadDao please help to reporduce the issue

/unassign

yanliang567 avatar Apr 17 '24 09:04 yanliang567

信件已收到      【自动回复】

dushulin avatar Apr 17 '24 09:04 dushulin

you can try to set the consistency_level param to Strong in the delete method

SimFG avatar Apr 17 '24 09:04 SimFG

you can try to set the consistency_level param to Strong in the delete method

how to call delete method by this param? I did not find api doc

dushulin avatar Apr 17 '24 10:04 dushulin

I saw a time traval value mentioned in your document. How long will it take before it will be retrieved by default?

dushulin avatar Apr 17 '24 10:04 dushulin

@dushulin maybe the go sdk have not give the param, you can set the collection consistency_level to strong, and try to delete the data.

SimFG avatar Apr 17 '24 13:04 SimFG

@dushulin maybe the go sdk have not give the param, you can set the collection consistency_level to strong, and try to delete the data. We need complex expr, so we can only set bounded,can we set consistency_level in query or search function? If not, can you team add this in go sdk?

dushulin avatar Apr 18 '24 01:04 dushulin

@congqixia please give this issue a look

SimFG avatar Apr 18 '24 03:04 SimFG

@dushulin you can set the consistency_level in search and query request https://github.com/milvus-io/milvus-sdk-go/blob/e697167f1828049c4c6237f0e3646b77b725ebdf/client/options.go#L174-L179

SimFG avatar Apr 18 '24 03:04 SimFG

@dushulin you can set the consistency_level in search and query request https://github.com/milvus-io/milvus-sdk-go/blob/e697167f1828049c4c6237f0e3646b77b725ebdf/client/options.go#L174-L179

Are there fields for curl?

dushulin avatar Apr 18 '24 06:04 dushulin

here is the client api

func (c *GrpcClient) Search(ctx context.Context, collName string, partitions []string,
    expr string, outputFields []string, vectors []entity.Vector, vectorField string, metricType entity.MetricType, topK int, sp entity.SearchParam, opts ...SearchQueryOptionFunc) ([]SearchResult, error) {

there is a SearchQueryOptionFunc param, it can be use the WithSearchQueryConsistencyLevel method to get this param

// WithSearchQueryConsistencyLevel specifies consistency level
func WithSearchQueryConsistencyLevel(cl entity.ConsistencyLevel) SearchQueryOptionFunc {
	return func(option *SearchQueryOption) {
		option.ConsistencyLevel = cl
	}
}

SimFG avatar Apr 18 '24 06:04 SimFG

here is the client api

func (c *GrpcClient) Search(ctx context.Context, collName string, partitions []string,
    expr string, outputFields []string, vectors []entity.Vector, vectorField string, metricType entity.MetricType, topK int, sp entity.SearchParam, opts ...SearchQueryOptionFunc) ([]SearchResult, error) {

there is a SearchQueryOptionFunc param, it can be use the WithSearchQueryConsistencyLevel method to get this param

// WithSearchQueryConsistencyLevel specifies consistency level
func WithSearchQueryConsistencyLevel(cl entity.ConsistencyLevel) SearchQueryOptionFunc {
	return func(option *SearchQueryOption) {
		option.ConsistencyLevel = cl
	}
}

I mean http api param name. ConsistencyLevel=Bounded? can you give me some example? use curl command

dushulin avatar Apr 18 '24 08:04 dushulin

@dushulin you can try to use the attu to delete the data. And the http api currently does not support this parameter, because it's a new parameter.

SimFG avatar Apr 18 '24 08:04 SimFG

sorry for the late reply. GoSDK use strong only for delete by expr.

congqixia avatar Apr 26 '24 06:04 congqixia

sorry for the late reply. GoSDK use strong only for delete by expr.

current GoSDK delete function does not has a ConsistencyLevel parameter. Do you mean the delete function use strong ConsistencyLevel by default?

phlamenco avatar Apr 28 '24 12:04 phlamenco

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

stale[bot] avatar Jun 02 '24 06:06 stale[bot]

信件已收到      【自动回复】

dushulin avatar Jun 02 '24 06:06 dushulin

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

stale[bot] avatar Jul 03 '24 02:07 stale[bot]

信件已收到      【自动回复】

dushulin avatar Jul 03 '24 02:07 dushulin

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

stale[bot] avatar Aug 04 '24 08:08 stale[bot]

信件已收到      【自动回复】

dushulin avatar Aug 04 '24 08:08 dushulin

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

stale[bot] avatar Sep 07 '24 19:09 stale[bot]

信件已收到      【自动回复】

dushulin avatar Sep 07 '24 19:09 dushulin