dolphinscheduler
dolphinscheduler copied to clipboard
feat: Add K8sClientPool Support
Purpose of the pull request
This project aims to add K8s connection pool support to DolphinScheduler to enhance the utilization of K8s connections in the worker module. It also enables the worker to perform operations on a single K8s cluster using a shared K8s connection. This pr is related to this issue https://github.com/apache/dolphinscheduler/issues/17128
Brief change log
Currently, two classes have been developed. Among them, K8sClientPool acts as a connection pool, maintaining a Map<String, KubernetesClient> where the cluster’s masterUrl is used to identify each cluster. Each cluster shares a single K8sClient. The other class, K8sExecutor, corresponds to the previous K8sUtils. However, instead of creating and maintaining a separate connection for each task as in the past, connections are now obtained from the connection pool.
As a connection pool, K8sClientPool implements connection creation, retrieval, graceful shutdown, and health checks for the pool. In the future, more features will be added based on the requirements of the production environment.
Verify this pull request
This pull request is code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(or)
Pull Request Notice
If your pull request contains incompatible change, you should also add it to docs/docs/en/guide/upgrade/incompatible.md
Thanks for opening this pull request! Please check out our contributing guidelines. (https://github.com/apache/dolphinscheduler/blob/dev/docs/docs/en/contribute/join/pull-request.md)
ok
Closing for inactivity.
