linkis icon indicating copy to clipboard operation
linkis copied to clipboard

[Feature] [Entrance] Entrance concurrency of running jobs should be optimized in case of multi instances

Open Alexkun opened this issue 1 year ago • 1 comments

Search before asking

  • [X] I had searched in the issues and found no similar feature requirement.

Problem Description

image As it shows above, param 'maxRunningJob' use param 'wds.linkis.rm.instance' as the default value. If there are n entrance nodes, the total running task num will be n * param 'wds.linkis.rm.instance'。 Entrance服务中maxRunningJob参数默认值取wds.linkis.rm.instance。在集群有n个entrance实例时,总的任务运行数为n * wds.linkis.rm.instance 。

Description

No response

Use case

No response

solutions

1, get the value of wds.linkis.rm.instance defined m, num of all entrance nodes in the cluster defined as n , the value of param 'maxRunningJob' is m / n defined as t 2, define the minimum running job param 'wds.linkis.entrance.runningjob.min' with default value 5. 3, the group set the minimum value between t and the value of 'wds.linkis.entrance.runningjob.min' . 1,用户配置wds.linkis.rm.instance 定义为m,集群所有entrance 数量定义为n,则初始最大运行任务数为m/n ,定义为t 2,定义entrance最小运行任务数 wds.linkis.entrance.runningjob.min = 5 3,用户队列group设置t和wds.linkis.entrance.runningjob.min中的最大值,作为最大运行数

Anything else

No response

Are you willing to submit a PR?

  • [ ] Yes I am willing to submit a PR!

Alexkun avatar Jul 15 '22 08:07 Alexkun

:blush: Welcome to the Apache Linkis (incubating) community!! We are glad that you are contributing by opening this issue.

Please make sure to include all the relevant context. We will be here shortly.

If you are interested in contributing to our website project, please let us know! You can check out our contributing guide on :point_right: How to Participate in Project Contribution.

WeChat Group:

image Mailing Lists:

name description Subscribe Unsubscribe archive
[email protected] community activity information subscribe unsubscribe archive

github-actions[bot] avatar Jul 15 '22 08:07 github-actions[bot]