dolphinscheduler icon indicating copy to clipboard operation
dolphinscheduler copied to clipboard

[Improvement][Master] Add master explicit overload control

Open pegasas opened this issue 1 year ago • 4 comments

Purpose of the pull request

resolve: https://github.com/apache/dolphinscheduler/issues/16032

Brief change log

Now master/worker are all based on cpu/memory control in protection. see https://github.com/apache/dolphinscheduler/blob/5463d0255a71212668606588c10189c2fb0185d2/dolphinscheduler-meter/src/main/java/org/apache/dolphinscheduler/meter/metrics/BaseServerLoadProtection.java#L38 When deployed in k8s, it will get cpu/memory from pod node instead of pod JVM. Add a explicit overload control like worker.

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(or)

If your pull request contain incompatible change, you should also add it to docs/docs/en/guide/upgrede/incompatible.md

pegasas avatar May 20 '24 05:05 pegasas

We shouldn't add control here, once a workflow is be loaded means it has get all resource.

https://github.com/apache/dolphinscheduler/blob/5463d0255a71212668606588c10189c2fb0185d2/dolphinscheduler-worker/src/main/java/org/apache/dolphinscheduler/server/worker/runner/WorkerTaskExecutorThreadPool.java#L55-L66 I saw worker control workload like this.

pegasas avatar May 20 '24 10:05 pegasas

aligned with worker like https://github.com/apache/dolphinscheduler/pull/16027/files

pegasas avatar May 23 '24 03:05 pegasas

This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 7 days if no further activity occurs.

github-actions[bot] avatar Sep 21 '24 00:09 github-actions[bot]

This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 7 days if no further activity occurs.

remove

pegasas avatar Sep 21 '24 02:09 pegasas