milvus icon indicating copy to clipboard operation
milvus copied to clipboard

enhance: Move sort stats task to sort compaction

Open xiaocai2333 opened this issue 5 months ago • 44 comments

issue: #42560

xiaocai2333 avatar Jun 05 '25 11:06 xiaocai2333

@xiaocai2333 cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

mergify[bot] avatar Jun 05 '25 12:06 mergify[bot]

@xiaocai2333 go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify[bot] avatar Jun 05 '25 12:06 mergify[bot]

@xiaocai2333 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Jun 05 '25 13:06 mergify[bot]

continue to bind text_match to the SortCompaction task; otherwise, once sorting is completed, loading in a growing manner will lead to the creation of numerous temporary text_match indexes.

czs007 avatar Jun 05 '25 14:06 czs007

@xiaocai2333 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Jun 09 '25 04:06 mergify[bot]

@xiaocai2333 go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify[bot] avatar Jun 09 '25 04:06 mergify[bot]

Codecov Report

Attention: Patch coverage is 64.53333% with 266 lines in your changes missing coverage. Please review.

Project coverage is 78.92%. Comparing base (ed9aa1d) to head (cf82c85). Report is 5 commits behind head on master.

Files with missing lines Patch % Lines
internal/datanode/compactor/sort_compaction.go 59.02% 117 Missing and 17 partials :warning:
internal/datacoord/compaction_policy_single.go 65.04% 33 Missing and 10 partials :warning:
internal/datacoord/import_util.go 64.70% 18 Missing and 6 partials :warning:
internal/datanode/util/util.go 40.00% 13 Missing and 2 partials :warning:
internal/datacoord/import_checker.go 57.57% 9 Missing and 5 partials :warning:
internal/datanode/services.go 15.38% 11 Missing :warning:
internal/datacoord/compaction_task_mix.go 79.31% 5 Missing and 1 partial :warning:
internal/datacoord/stats_task_meta.go 40.00% 5 Missing and 1 partial :warning:
internal/datacoord/meta.go 44.44% 4 Missing and 1 partial :warning:
internal/datacoord/task_stats.go 77.77% 2 Missing :warning:
... and 6 more
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #42562      +/-   ##
==========================================
- Coverage   79.00%   78.92%   -0.08%     
==========================================
  Files        1560     1562       +2     
  Lines      222449   222844     +395     
==========================================
+ Hits       175735   175885     +150     
- Misses      40364    40569     +205     
- Partials     6350     6390      +40     
Components Coverage Δ
Client 79.39% <ø> (ø)
Core 73.81% <ø> (ø)
Go 79.90% <64.48%> (-0.10%) :arrow_down:
Files with missing lines Coverage Δ
internal/compaction/params.go 94.73% <100.00%> (+0.39%) :arrow_up:
internal/datacoord/compaction_inspector.go 67.96% <100.00%> (+1.99%) :arrow_up:
internal/datacoord/compaction_task.go 100.00% <ø> (ø)
internal/datacoord/compaction_task_clustering.go 73.40% <100.00%> (+0.23%) :arrow_up:
internal/datacoord/compaction_task_l0.go 73.81% <ø> (-0.65%) :arrow_down:
internal/datacoord/compaction_trigger.go 85.63% <100.00%> (-0.52%) :arrow_down:
internal/datacoord/import_inspector.go 82.19% <100.00%> (ø)
internal/datacoord/import_task.go 94.11% <100.00%> (ø)
internal/datacoord/import_task_import.go 83.73% <100.00%> (ø)
internal/datacoord/segment_info.go 86.17% <ø> (-0.35%) :arrow_down:
... and 27 more

... and 32 files with indirect coverage changes

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Jun 09 '25 07:06 codecov[bot]

@xiaocai2333 go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify[bot] avatar Jun 09 '25 08:06 mergify[bot]

@xiaocai2333 go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify[bot] avatar Jun 09 '25 11:06 mergify[bot]

@xiaocai2333 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Jun 09 '25 13:06 mergify[bot]

@xiaocai2333 go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify[bot] avatar Jun 09 '25 15:06 mergify[bot]

@xiaocai2333 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Jun 09 '25 15:06 mergify[bot]

@xiaocai2333 go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify[bot] avatar Jun 10 '25 05:06 mergify[bot]

@xiaocai2333 go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify[bot] avatar Jun 10 '25 09:06 mergify[bot]

@xiaocai2333 go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify[bot] avatar Jun 10 '25 12:06 mergify[bot]

@xiaocai2333 cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

mergify[bot] avatar Jun 10 '25 14:06 mergify[bot]

@xiaocai2333 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Jun 10 '25 16:06 mergify[bot]

@xiaocai2333 go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify[bot] avatar Jun 11 '25 03:06 mergify[bot]

@xiaocai2333 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Jun 11 '25 03:06 mergify[bot]

rerun go-sdk

xiaocai2333 avatar Jun 11 '25 05:06 xiaocai2333

/run-cpu-e2e

xiaocai2333 avatar Jun 11 '25 05:06 xiaocai2333

@xiaocai2333 go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify[bot] avatar Jun 11 '25 06:06 mergify[bot]

@xiaocai2333 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Jun 11 '25 06:06 mergify[bot]

@xiaocai2333 go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify[bot] avatar Jun 11 '25 09:06 mergify[bot]

@xiaocai2333 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] avatar Jun 11 '25 12:06 mergify[bot]

/run-cpu-e2e

xiaocai2333 avatar Jun 11 '25 12:06 xiaocai2333

@xiaocai2333 go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify[bot] avatar Jun 11 '25 14:06 mergify[bot]

rerun go-sdk

xiaocai2333 avatar Jun 12 '25 01:06 xiaocai2333

/run-cpu-e2e

xiaocai2333 avatar Jun 12 '25 01:06 xiaocai2333

/run-cpu-e2e

xiaocai2333 avatar Jun 12 '25 04:06 xiaocai2333