feat: add configurable batch factor and runtime check bypass for embedding functions

[junjiejiangjjj]

https://github.com/milvus-io/milvus/issues/45544

• Add batch_factor configuration parameter (default: 5) to control embedding provider batch sizes
• Add disable_func_runtime_check property to bypass function validation during collection creation
• Add database interceptor support for AddCollectionFunction, AlterCollectionFunction, and DropCollectionFunction requests

[sre-ci-robot]

[ci-v2-notice] Notice: We are gradually rolling out the new ci-v2 system.

• Legacy CI jobs remain unaffected, you can just ignore ci-v2 if you don't want to run it.
• Additional "ci-v2/*" checkers will run for this PR to ensure the new ci-v2 system is working as expected.
• For tests that exist in both v1 and v2, passing in either system is considered PASS.

To rerun ci-v2 checks, comment with:

• /ci-rerun-code-check // for ci-v2/code-check
• /ci-rerun-build // for ci-v2/build
• /ci-rerun-ut-integration // for ci-v2/ut-integration
• /ci-rerun-ut-go // for ci-v2/ut-go
• /ci-rerun-ut-cpp // for ci-v2/ut-cpp
• /ci-rerun-ut // for all ci-v2/ut-integration, ci-v2/ut-go, ci-v2/ut-cpp
• /ci-rerun-e2e-arm // for ci-v2/e2e-arm

If you have any questions or requests, please contact @zhikunyao.

[mergify[bot]]

@junjiejiangjjj Please associate the related issue to the body of your Pull Request. (eg. "issue: #")

[codecov[bot]]

Codecov Report

:x: Patch coverage is 78.46154% with 14 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 76.49%. Comparing base (aa0870d) to head (8c06c58). :warning: Report is 5 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/common/common.go	33.33%	5 Missing and 1 partial :warning:
internal/proxy/task.go	50.00%	2 Missing and 1 partial :warning:
pkg/util/paramtable/function_param.go	76.92%	2 Missing and 1 partial :warning:
internal/proxy/function_task.go	0.00%	0 Missing and 2 partials :warning:

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #45592       +/-   ##
===========================================
- Coverage   83.19%   76.49%    -6.71%     
===========================================
  Files         521     1875     +1354     
  Lines       81430   292668   +211238     
===========================================
+ Hits        67744   223870   +156126     
- Misses      13686    61362    +47676     
- Partials        0     7436     +7436     

Components	Coverage Δ
Client	78.17% <ø> (∅)
Core	83.19% <ø> (ø)
Go	74.60% <78.46%> (∅)

Files with missing lines	Coverage Δ
internal/datacoord/server.go	67.88% <100.00%> (ø)
internal/proxy/database_interceptor.go	93.66% <100.00%> (ø)
internal/proxy/util.go	80.92% <100.00%> (ø)
.../util/function/embedding/ali_embedding_provider.go	81.92% <100.00%> (ø)
...l/function/embedding/bedrock_embedding_provider.go	82.08% <100.00%> (ø)
...il/function/embedding/cohere_embedding_provider.go	85.00% <100.00%> (ø)
...il/function/embedding/openai_embedding_provider.go	78.09% <100.00%> (ø)
...nction/embedding/siliconflow_embedding_provider.go	79.66% <100.00%> (ø)
.../util/function/embedding/tei_embedding_provider.go	86.04% <100.00%> (ø)
...util/function/embedding/text_embedding_function.go	88.28% <100.00%> (ø)
... and 8 more

... and 1336 files with indirect coverage changes

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[mergify[bot]]

@junjiejiangjjj go-sdk check failed, comment rerun go-sdk can trigger the job again.

[mergify[bot]]

@junjiejiangjjj cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

[junjiejiangjjj]

/run-cpu-e2e

[junjiejiangjjj]

rerun go-sdk

[mergify[bot]]

@junjiejiangjjj cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

[mergify[bot]]

@junjiejiangjjj Thanks for your contribution. Please submit with DCO, see the contributing guide https://github.com/milvus-io/milvus/blob/master/CONTRIBUTING.md#developer-certificate-of-origin-dco.

[mergify[bot]]

@junjiejiangjjj cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

[mergify[bot]]

@junjiejiangjjj cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

[sre-ci-robot]

[ci-v2-notice] Notice: We are gradually rolling out the new ci-v2 system.

• Legacy CI jobs remain unaffected, you can just ignore ci-v2 if you don't want to run it.
• Additional "ci-v2/*" checkers will run for this PR to ensure the new ci-v2 system is working as expected.
• For tests that exist in both v1 and v2, passing in either system is considered PASS.

To rerun ci-v2 checks, comment with:

• /ci-rerun-code-check // for ci-v2/code-check
• /ci-rerun-build // for ci-v2/build
• /ci-rerun-ut-integration // for ci-v2/ut-integration
• /ci-rerun-ut-go // for ci-v2/ut-go
• /ci-rerun-ut-cpp // for ci-v2/ut-cpp
• /ci-rerun-ut // for all ci-v2/ut-integration, ci-v2/ut-go, ci-v2/ut-cpp
• /ci-rerun-e2e-arm // for ci-v2/e2e-arm

If you have any questions or requests, please contact @zhikunyao.

[mergify[bot]]

@junjiejiangjjj cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

[junjiejiangjjj]

/ci-rerun-ut-integration

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: junjiejiangjjj, liliu-z

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [liliu-z]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment