[DSIP-27][Task Plugin] Some improvements of JAVA task plugin

[ailiujiarui]

Purpose of the pull request

Update the Java task plugin close:https://github.com/apache/dolphinscheduler/issues/15819

Brief change log

• Deprecate write java code in JAVA task
• Rename the JAR to FATJAR Only the displayed name has been modified, the functionality remains unchanged.
• Add the new type NORMALJAR Allow users to upload normal type of jar files, which require external libraries to run properly. Users do not need to package all libraries and runtime files into a fat jar each time, making the submission of jar files more flexible and convenient.
• Update the tests Delete tests about JAVA type and add the test of NORMAL JAR type
• Update the Javadoc comment
• Remove the Java task type tests from the e2e tests Since the original workflowJavaTaskE2ETest was used to test Java code types under Java task types, and this feature has been removed, this test is no longer needed.

type of FATJAR

type of NORMALJAR

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(or)

Pull Request Notice

Pull Request Notice

If your pull request contain incompatible change, you should also add it to docs/docs/en/guide/upgrede/incompatible.md

[fuchanghai]

Please check if additional explanation is needed

[ailiujiarui]

Hi everyone, could you please let me know if there’s anything that needs improvement? Thanks!

[ailiujiarui]

I am not sure what caused the bc38b84 CI test error

[ailiujiarui]

请先解决冲突。

Conflict has been resolved.

[SbloodyS]

Document has been updated.

I didn't see any docs in this PR. Please check it.

[ailiujiarui]

Please change the docs too.

docs has been updated

[Gallardot]

Please don't add .jar in the source code, the .jar file will be excluded in src, these means the e2e will not be executed after release.

+1. I suggest adding the source code of the test jar to e2e module as a submodule, and compiling the corresponding jar through mvn package for e2e to test

cc: @ailiujiarui @SbloodyS @ruanwenjun

[SbloodyS]

Please don't add .jar in the source code, the .jar file will be excluded in src, these means the e2e will not be executed after release.

+1. I suggest adding the source code of the test jar to e2e module as a submodule, and compiling the corresponding jar through mvn package for e2e to test

cc: @ailiujiarui @SbloodyS @ruanwenjun

Agreed.

[ailiujiarui]

Please don't add .jar in the source code, the .jar file will be excluded in src, these means the e2e will not be executed after release.请不要在源码中添加 .jar，.jar 文件将被排除在 src 中，这意味着 e2e 发布后不会执行。

+1. I suggest adding the source code of the test jar to e2e module as a submodule, and compiling the corresponding jar through mvn package for e2e to test+1.我建议将测试 jar 的源码作为子模块添加到 e2e 模块中，并通过 mvn 包编译相应的 jar 供 e2e 测试 cc:  抄送：@ailiujiarui @SbloodyS @ruanwenjun

Agreed. 同意。

I first tried placing the three classes fat, normal1, and normal2 in the common package under e2e-case, then compiled and packaged them into JAR files through code to provide for other workflows. However, the problem arose because the worker uses Java 8, while e2etest uses Java 11. The JAR packaged under the e2e directory couldn't be executed by the worker, throwing an error due to a higher version. I then tried moving common to the resources package under e2e-case and created a shell workflow to compile these three classes using the worker and output them to /tmp. The code then packages them into a JAR, but the current issue is that it keeps reporting that it can't find Fat.class under /tmp, resulting in an empty JAR. I don't know what the reason is.

[ailiujiarui]

The workload for this end-to-end test case is much greater than I initially anticipated. Would it be possible to focus on unit testing in the current pull request? We can add the end-to-end tests later on. @SbloodyS @ruanwenjun

[SbloodyS]

The workload for this end-to-end test case is much greater than I initially anticipated. Would it be possible to focus on unit testing in the current pull request? We can add the end-to-end tests later on. @SbloodyS @ruanwenjun

I don't think it's a good idea. I'm -1 on this.

[pinkfloyds]

Don't be discouraged. Keep going and keep up the good work.

[ailiujiarui]

Please don't add .jar in the source code, the .jar file will be excluded in src, these means the e2e will not be executed after release.请不要在源码中添加 .jar，.jar 文件将被排除在 src 中，这意味着 e2e 发布后不会执行。

I implemented a strategy where the source code is compiled and packaged into a JAR file during the application's runtime. This approach eliminates the necessity of mounting files within the image.

[SbloodyS]

There are some CI failed. Please check it. @ailiujiarui

[ailiujiarui]

There are some CI failed. Please check it. @ailiujiarui

The bug has been fixed

[SbloodyS]

There are some CI failed. Please check it. @ailiujiarui

The bug has been fixed

The WorkflowJavaTaskE2ETest still failed.

[ailiujiarui]

There are some CI failed. Please check it. @ailiujiarui

The bug has been fixed

The WorkflowJavaTaskE2ETest still failed.

I've encountered a similar issue before as well. What worked for me was rerunning the entire test from the beginning, and that allowed it to pass successfully.

[SbloodyS]

I've encountered a similar issue before as well. What worked for me was rerunning the entire test from the beginning, and that allowed it to pass successfully.

You should check the error message of java.lang.RuntimeException: No such package: normal1.jar at org.apache.dolphinscheduler.e2e.cases.WorkflowJavaTaskE2ETest.testCreateNormalJarWorkflow(WorkflowJavaTaskE2ETest.java:341)

[ailiujiarui]

I've encountered a similar issue before as well. What worked for me was rerunning the entire test from the beginning, and that allowed it to pass successfully.

You should check the error message of java.lang.RuntimeException: No such package: normal1.jar at org.apache.dolphinscheduler.e2e.cases.WorkflowJavaTaskE2ETest.testCreateNormalJarWorkflow(WorkflowJavaTaskE2ETest.java:341)

Through testing, it was found that unstable file uploads—often resulting in failed uploads—and frequently missing resource file selections in the Resources panel are caused by the component's loading process. Specifically, webwait only detects the presence of DOM elements without ensuring that their child elements have fully loaded. In other words, the code continues to execute before the page is completely rendered, leading to these issues.

To address this problem, I implemented thread.sleep to wait for the complete loading. This method is currently stable and effective. I also considered other approaches, but none could effectively and concisely replace it.

[ailiujiarui]

Requires Approval @ruanwenjun

[sonarqubecloud[bot]]

Please retry analysis of this Pull-Request directly on SonarQube Cloud

[boring-cyborg[bot]]

Awesome work, congrats on your first merged pull request!

[ruanwenjun]

Merged into dev, will be released at 3.3.0, thanks for your PR, helps a lot. @ailiujiarui