dolphinscheduler icon indicating copy to clipboard operation
dolphinscheduler copied to clipboard

feat: Add K8sClientPool Support

Open PengJingzhao opened this issue 7 months ago • 3 comments

Purpose of the pull request

This project aims to add K8s connection pool support to DolphinScheduler to enhance the utilization of K8s connections in the worker module. It also enables the worker to perform operations on a single K8s cluster using a shared K8s connection. This pr is related to this issue https://github.com/apache/dolphinscheduler/issues/17128

Brief change log

Currently, two classes have been developed. Among them, K8sClientPool acts as a connection pool, maintaining a Map<String, KubernetesClient> where the cluster’s masterUrl is used to identify each cluster. Each cluster shares a single K8sClient. The other class, K8sExecutor, corresponds to the previous K8sUtils. However, instead of creating and maintaining a separate connection for each task as in the past, connections are now obtained from the connection pool.

As a connection pool, K8sClientPool implements connection creation, retrieval, graceful shutdown, and health checks for the pool. In the future, more features will be added based on the requirements of the production environment.

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(or)

Pull Request Notice

Pull Request Notice

If your pull request contains incompatible change, you should also add it to docs/docs/en/guide/upgrade/incompatible.md

PengJingzhao avatar Apr 13 '25 12:04 PengJingzhao

Thanks for opening this pull request! Please check out our contributing guidelines. (https://github.com/apache/dolphinscheduler/blob/dev/docs/docs/en/contribute/join/pull-request.md)

boring-cyborg[bot] avatar Apr 13 '25 12:04 boring-cyborg[bot]

ok

PengJingzhao avatar Apr 14 '25 02:04 PengJingzhao

Quality Gate Failed Quality Gate failed

Failed conditions
10.8% Coverage on New Code (required ≥ 60%)

See analysis details on SonarQube Cloud

sonarqubecloud[bot] avatar Apr 17 '25 02:04 sonarqubecloud[bot]

Closing for inactivity.

SbloodyS avatar Aug 06 '25 01:08 SbloodyS