linkis
linkis copied to clipboard
[Feature][1.5.0] Linkis orchestrator module supports plugin configuration (#5082)
What is the purpose of the change
In some scenarios, the Linkis orchestrator module is not required, but the orchestrator does not support plugin configuration. Linkis orchestrator supports the use of plugins, and it is also possible to bypass Linkis orchestrator.
Related issues/PRs
Related issues: #5082
Brief change log
- Define linkis.entrance.skip.orchestrator parameter to skip linkis orchestrator when needed;
- Define CodeLogicalUnitSimpleExecTask class to bypass linkis orchestrator module to execute simple task.
Checklist
- [x] I have read the Contributing Guidelines on pull requests.
- [x] I have explained the need for this PR and the problem it solves
- [x] I have explained the changes or the new features added to this PR
- [x] I have added tests corresponding to this change
- [x] I have updated the documentation to reflect this change
- [x] I have verified that this change is backward compatible (If not, please discuss on the Linkis mailing list first)
- [x] If this is a code change: I have written unit tests to fully verify the new behavior.
Test Result
Execute hive task:
Execute shell task:
ping~ @guoshupei
It looks good to me, there are a few points that can be further optimized:
-
Need to consider the update of log and progress information dealLog/dealProgressWithResource
-
Whether classes related to org.apache.linkis.orchestrator.ecm can be reused
-
callSimpleExecute needs to consider returning an asynchronous response and asynchronous notification of task completion.
-
Need to support retry scenario
-
需要考虑日志和进度信息的更新dealLog/dealProgressWithResource
-
org.apache.linkis.orchestrator.ecm相关的类是否可以复用
-
callSimpleExecute需要考虑返回异步响应,任务异步通知完成的方式
-
需要支持retry场景
Repeated PR #5120