dolphinscheduler icon indicating copy to clipboard operation
dolphinscheduler copied to clipboard

Replace K8S task watcher to K8S task informer

Open zhaohehuhu opened this issue 2 years ago • 7 comments

Purpose of the pull request

I'd like to replace K8S task watcher to informer. Sometimes watcher may not receive full watcher events for a Job, which may make K8S task stuck in the wrong status(See issue) Besides, compared with watcher, informer can get all events that job had even before informer was running. Informer provides a better way to cache and retrieve resources from the Kubernetes API server efficiently based on watcher, which can highly reduce API server workload.

Brief change log

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(or)

If your pull request contain incompatible change, you should also add it to docs/docs/en/guide/upgrede/incompatible.md

zhaohehuhu avatar Oct 20 '23 06:10 zhaohehuhu

Hi @zhaohehuhu, you should better link an issue.

Radeity avatar Oct 21 '23 03:10 Radeity

@Gallardot PTAL

Radeity avatar Oct 21 '23 04:10 Radeity

Is there any way to add K8sTask test case?

ruanwenjun avatar Oct 24 '23 04:10 ruanwenjun

Is there any way to add K8sTask test case?

+1. Adding test cases for K8sTask is highly necessary.

Additionally, I am not very familiar with the difference between watcher and informer in the k8s java-client. Can you explain the difference between watcher and informer, or provide some reference documents? @zhaohehuhu

Gallardot avatar Oct 24 '23 09:10 Gallardot

Is there any way to add K8sTask test case?

There is already a UT for K8sTask. But it's not easy to do a full test case that covers every things in K8S task.

zhaohehuhu avatar Oct 26 '23 09:10 zhaohehuhu

Is there any way to add K8sTask test case?

+1. Adding test cases for K8sTask is highly necessary.

Additionally, I am not very familiar with the difference between watcher and informer in the k8s java-client. Can you explain the difference between watcher and informer, or provide some reference documents? @zhaohehuhu

I can provide the doc. But you can click on the issue I mentioned earlier for details now.

zhaohehuhu avatar Oct 26 '23 09:10 zhaohehuhu

This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 7 days if no further activity occurs.

github-actions[bot] avatar May 10 '24 00:05 github-actions[bot]

This pull request has been closed because it has not had recent activity. You could reopen it if you try to continue your work, and anyone who are interested in it are encouraged to continue work on this pull request.

github-actions[bot] avatar May 17 '24 00:05 github-actions[bot]