milvus icon indicating copy to clipboard operation
milvus copied to clipboard

Support Bulk Load in master branch

Open soothing-rain opened this issue 3 years ago • 43 comments

soothing-rain avatar Sep 02 '22 03:09 soothing-rain

/kind improvement /hold

soothing-rain avatar Sep 02 '22 03:09 soothing-rain

@soothing-rain, please be sure the pr should only have one commit, check https://github.com/milvus-io/milvus/blob/master/CODE_REVIEW.md for more details.

mergify[bot] avatar Sep 02 '22 03:09 mergify[bot]

@soothing-rain ut workflow job failed, comment rerun ut can trigger the job again.

mergify[bot] avatar Sep 02 '22 03:09 mergify[bot]

@soothing-rain E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Sep 02 '22 04:09 mergify[bot]

@soothing-rain ut workflow job failed, comment rerun ut can trigger the job again.

mergify[bot] avatar Sep 02 '22 08:09 mergify[bot]

Codecov Report

Merging #18982 (83c1d02) into master (c15b880) will increase coverage by 0.10%. The diff coverage is 73.01%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #18982      +/-   ##
==========================================
+ Coverage   80.76%   80.86%   +0.10%     
==========================================
  Files         667      667              
  Lines       93705    94110     +405     
==========================================
+ Hits        75678    76104     +426     
+ Misses      14822    14744      -78     
- Partials     3205     3262      +57     
Impacted Files Coverage Δ
internal/datacoord/cluster.go 95.45% <ø> (-0.11%) :arrow_down:
internal/datacoord/handler.go 78.72% <0.00%> (-1.72%) :arrow_down:
internal/datacoord/server.go 77.37% <ø> (+1.24%) :arrow_up:
internal/datacoord/session_manager.go 84.78% <ø> (+0.20%) :arrow_up:
internal/datanode/binlog_io.go 93.66% <ø> (ø)
internal/distributed/proxy/service.go 74.82% <ø> (ø)
internal/proxy/impl.go 90.33% <ø> (-0.06%) :arrow_down:
internal/util/paramtable/component_param.go 87.51% <ø> (-0.07%) :arrow_down:
internal/rootcoord/broker.go 94.59% <11.11%> (-5.41%) :arrow_down:
internal/util/importutil/json_handler.go 81.65% <30.76%> (-3.48%) :arrow_down:
... and 57 more

codecov[bot] avatar Sep 02 '22 09:09 codecov[bot]

@soothing-rain ut workflow job failed, comment rerun ut can trigger the job again.

mergify[bot] avatar Sep 02 '22 10:09 mergify[bot]

@soothing-rain ut workflow job failed, comment rerun ut can trigger the job again.

mergify[bot] avatar Sep 02 '22 10:09 mergify[bot]

@soothing-rain ut workflow job failed, comment rerun ut can trigger the job again.

mergify[bot] avatar Sep 07 '22 06:09 mergify[bot]

@soothing-rain ut workflow job failed, comment rerun ut can trigger the job again.

mergify[bot] avatar Sep 07 '22 17:09 mergify[bot]

@soothing-rain E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Sep 07 '22 17:09 mergify[bot]

@soothing-rain E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Sep 09 '22 04:09 mergify[bot]

/assign @xiaofan-luan

soothing-rain avatar Sep 09 '22 07:09 soothing-rain

When we do. import.

  1. we get collectionId by collection name from cache
  2. we getVchannels from collID (which I don't think is correct),because channels and partition does't has any relations.

what we should to is to send default value to root and let root to fit in all the partition values, proxy shouldn't handle this

xiaofan-luan avatar Sep 12 '22 15:09 xiaofan-luan

When we do. import.

  1. we get collectionId by collection name from cache
  2. we getVchannels from collID (which I don't think is correct),because channels and partition does't has any relations.

what we should to is to send default value to root and let root to fit in all the partition values, proxy shouldn't handle this

done

soothing-rain avatar Sep 13 '22 12:09 soothing-rain

@soothing-rain ut workflow job failed, comment rerun ut can trigger the job again.

mergify[bot] avatar Sep 13 '22 13:09 mergify[bot]

rerun ut

soothing-rain avatar Sep 13 '22 15:09 soothing-rain

@soothing-rain ut workflow job failed, comment rerun ut can trigger the job again.

mergify[bot] avatar Sep 13 '22 16:09 mergify[bot]

@soothing-rain ut workflow job failed, comment rerun ut can trigger the job again.

mergify[bot] avatar Sep 14 '22 12:09 mergify[bot]

rerun ut

soothing-rain avatar Sep 15 '22 02:09 soothing-rain

@soothing-rain E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Sep 15 '22 14:09 mergify[bot]

@soothing-rain ut workflow job failed, comment rerun ut can trigger the job again.

mergify[bot] avatar Sep 15 '22 14:09 mergify[bot]

rerun ut

soothing-rain avatar Sep 15 '22 15:09 soothing-rain

@soothing-rain ut workflow job failed, comment rerun ut can trigger the job again.

mergify[bot] avatar Sep 15 '22 16:09 mergify[bot]

rerun ut

soothing-rain avatar Sep 15 '22 16:09 soothing-rain

@soothing-rain Thanks for your contribution. Please submit with DCO, see the contributing guide https://github.com/milvus-io/milvus/blob/master/CONTRIBUTING.md#developer-certificate-of-origin-dco.

mergify[bot] avatar Sep 20 '22 08:09 mergify[bot]

@soothing-rain ut workflow job failed, comment rerun ut can trigger the job again.

mergify[bot] avatar Sep 20 '22 08:09 mergify[bot]

@soothing-rain Thanks for your contribution. Please submit with DCO, see the contributing guide https://github.com/milvus-io/milvus/blob/master/CONTRIBUTING.md#developer-certificate-of-origin-dco.

mergify[bot] avatar Sep 20 '22 10:09 mergify[bot]

@soothing-rain ut workflow job failed, comment rerun ut can trigger the job again.

mergify[bot] avatar Sep 20 '22 10:09 mergify[bot]

@soothing-rain E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Sep 20 '22 10:09 mergify[bot]