linkis icon indicating copy to clipboard operation
linkis copied to clipboard

[Feature][1.5.0] Linkis orchestrator module supports plugin configuration (#5082)

Open yuechao24 opened this issue 1 year ago • 2 comments

What is the purpose of the change

In some scenarios, the Linkis orchestrator module is not required, but the orchestrator does not support plugin configuration. Linkis orchestrator supports the use of plugins, and it is also possible to bypass Linkis orchestrator.

Related issues/PRs

Related issues: #5082

Brief change log

  • Define linkis.entrance.skip.orchestrator parameter to skip linkis orchestrator when needed;
  • Define CodeLogicalUnitSimpleExecTask class to bypass linkis orchestrator module to execute simple task.

Checklist

  • [x] I have read the Contributing Guidelines on pull requests.
  • [x] I have explained the need for this PR and the problem it solves
  • [x] I have explained the changes or the new features added to this PR
  • [x] I have added tests corresponding to this change
  • [x] I have updated the documentation to reflect this change
  • [x] I have verified that this change is backward compatible (If not, please discuss on the Linkis mailing list first)
  • [x] If this is a code change: I have written unit tests to fully verify the new behavior.

Test Result

Execute hive task: image

Execute shell task: image

yuechao24 avatar Feb 07 '24 10:02 yuechao24

ping~ @guoshupei

peacewong avatar Feb 19 '24 13:02 peacewong

It looks good to me, there are a few points that can be further optimized:

  1. Need to consider the update of log and progress information dealLog/dealProgressWithResource

  2. Whether classes related to org.apache.linkis.orchestrator.ecm can be reused

  3. callSimpleExecute needs to consider returning an asynchronous response and asynchronous notification of task completion.

  4. Need to support retry scenario

  5. 需要考虑日志和进度信息的更新dealLog/dealProgressWithResource

  6. org.apache.linkis.orchestrator.ecm相关的类是否可以复用

  7. callSimpleExecute需要考虑返回异步响应,任务异步通知完成的方式

  8. 需要支持retry场景

peacewong avatar Feb 20 '24 04:02 peacewong

Repeated PR #5120

peacewong avatar Jun 05 '24 07:06 peacewong