milvus icon indicating copy to clipboard operation
milvus copied to clipboard

Optimize large memory usage of InsertRecord by using vector instead of unordered_map if InsertRecord used in sealed segment

Open aoiasd opened this issue 3 years ago • 26 comments

related:https://github.com/milvus-io/milvus/issues/18764 https://github.com/milvus-io/milvus/issues/18845 Signed-off-by: aoiasd [email protected]

aoiasd avatar Sep 16 '22 08:09 aoiasd

@aoiasd E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Sep 16 '22 08:09 mergify[bot]

@aoiasd E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Sep 16 '22 09:09 mergify[bot]

Codecov Report

Merging #19245 (44cdef3) into master (72ee35a) will increase coverage by 0.02%. The diff coverage is 93.96%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #19245      +/-   ##
==========================================
+ Coverage   80.25%   80.28%   +0.02%     
==========================================
  Files         673      673              
  Lines       94174    94203      +29     
==========================================
+ Hits        75581    75632      +51     
+ Misses      15326    15313      -13     
+ Partials     3267     3258       -9     
Impacted Files Coverage Δ
internal/core/src/query/SearchOnSealed.cpp 100.00% <ø> (ø)
internal/core/src/segcore/FieldIndexing.cpp 97.05% <ø> (-0.38%) :arrow_down:
internal/core/src/segcore/SegmentGrowingImpl.h 82.85% <ø> (ø)
internal/core/src/segcore/SegmentSealedImpl.cpp 79.79% <ø> (ø)
internal/core/src/segcore/SegmentSealedImpl.h 80.00% <ø> (ø)
internal/core/src/segcore/Utils.cpp 70.96% <ø> (-3.20%) :arrow_down:
internal/core/src/segcore/InsertRecord.h 92.17% <91.66%> (-7.83%) :arrow_down:
internal/core/src/segcore/FieldIndexing.h 97.61% <100.00%> (+0.74%) :arrow_up:
internal/core/src/segcore/Utils.h 100.00% <100.00%> (ø)
... and 19 more

codecov[bot] avatar Sep 16 '22 10:09 codecov[bot]

@aoiasd ut workflow job failed, comment rerun ut can trigger the job again.

mergify[bot] avatar Sep 16 '22 11:09 mergify[bot]

rerun ut

aoiasd avatar Sep 19 '22 02:09 aoiasd

/run-cpu-e2e

aoiasd avatar Sep 19 '22 02:09 aoiasd

@aoiasd E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Sep 19 '22 02:09 mergify[bot]

/run-cpu-e2e

aoiasd avatar Sep 19 '22 02:09 aoiasd

@aoiasd E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Sep 19 '22 02:09 mergify[bot]

/run-cpu-e2e

aoiasd avatar Sep 19 '22 04:09 aoiasd

@aoiasd E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Sep 19 '22 04:09 mergify[bot]

/run-cpu-e2e

aoiasd avatar Sep 19 '22 08:09 aoiasd

@aoiasd E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Sep 19 '22 08:09 mergify[bot]

/run-cpu-e2e

aoiasd avatar Sep 19 '22 08:09 aoiasd

@aoiasd E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Sep 19 '22 08:09 mergify[bot]

/run-cpu-e2e

aoiasd avatar Sep 19 '22 08:09 aoiasd

@aoiasd E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Sep 19 '22 09:09 mergify[bot]

@aoiasd E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Sep 20 '22 02:09 mergify[bot]

/run-cpu-e2e

aoiasd avatar Sep 20 '22 03:09 aoiasd

@aoiasd E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Sep 20 '22 03:09 mergify[bot]

/run-cpu-e2e

aoiasd avatar Sep 20 '22 08:09 aoiasd

@aoiasd E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Sep 21 '22 07:09 mergify[bot]

/run-cpu-e2e

aoiasd avatar Sep 21 '22 07:09 aoiasd

/lgtm

jiaoew1991 avatar Sep 22 '22 01:09 jiaoew1991

/lgtm

jiaoew1991 avatar Sep 23 '22 06:09 jiaoew1991

/approve

congqixia avatar Sep 23 '22 09:09 congqixia

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aoiasd, congqixia

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

sre-ci-robot avatar Sep 23 '22 09:09 sre-ci-robot