dolphinscheduler
dolphinscheduler copied to clipboard
[DSIP-27] Deprecate write java code in JAVA task
Search before asking
- [X] I had searched in the issues and found no similar feature requirement.
Motivation
Right now, we support two kind of usage of JAVA tasks,
The first type is JAVA, user can write java code, once the worker receive this kind of task, it will compile the java code to class, and then execute the class file.
The second type is JAR, user can submit a fat jar, once the worker receive this kind of task, it will directly execute the fat jar.
This issue is want to remove the first type of JAVA task, since this is more like a demo, no one will submit java code, and in most of the time, the java application is not written within one file.
So we only need to support submit Jar.
Design Detail
Further more, we would better support user submit jars rather than a fat jar. The command may look like below, this should be more usefully
java $JAVA_OPTS \
-Dxx.yy.args=1 \
-cp "xx.jar":"xx.conf" \
xx.Application args
So we need to change the current task mode into: Fat JAR
, NORMAL JAR
Compatibility, Deprecation, and Migration Plan
This is not compatible with the current version.
Test Plan
Add unit test.
Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
LGTM
The DSIP has been added to OSPP, no further development needs to be specified.
How about this
- Remove the selection boxes for Run Type, and add dependency package parameters. If this parameter is empty, it means fat jar, and if it is not empty, it means normal jar.
- Add Main class parameters.
The DSIP has been added to OSPP, no further development needs to be specified.
Why can this be added to OSPP so casually? OSPP has a very long end time, which will block releases.
The DSIP has been added to OSPP, no further development needs to be specified.
Why can this be added to OSPP so casually? OSPP has a very long end time, which will block releases.
I apologize for not explaining in advance. Initially, I thought it wasn't a highly important feature, just a few versions of delay.
LGTM, In addition, We should be cautious about adding new tasks in the future, and try to receive enough votes and requests to add new tasks.
Please assign to me
I am a student selected in this OSPP, and here is part of my implementation plan.
1. Delete the Java type
If you want to add a new task type, you need to first delete the related code of the Java type from the previous task types.
For example, this piece of code first determines whether it is a Java type. You can use this as a starting point to delete the JAVA type and leave a mark. Later, you can add the Normal Jar type. By the same token, in all related places, first delete the code of the Java type task, including calls, uses, etc. By the way, record the position. After deletion is complete and other functions are normal, add the Normal jar type at these recorded places.
2. Partial implementation of the Normal Jar type
First, we need to add a new constant in the JavaConstants class to represent the NORMAL JAR task type:
// Original code
public class JavaConstants {
public static final String RUN_TYPE_JAVA = "JAVA";
public static final String RUN_TYPE_JAR = "JAR";
}
// New code
public class JavaConstants {
public static final String RUN_TYPE_JAVA = "JAVA";
public static final String RUN_TYPE_JAR = "JAR";
public static final String RUN_TYPE_NORMAL_JAR = "NORMAL_JAR"; // New constant
}
Then, update the checkParameters method in the JavaParameters class to accept the new run type:
// Original code
public class JavaParameters extends AbstractParameters {
// ...
public boolean checkParameters() {
return runType != null && (runType.equals(JavaConstants.RUN_TYPE_JAVA) || runType.equals(JavaConstants.RUN_TYPE_JAR));
}
}
// New code
public class JavaParameters extends AbstractParameters {
// ...
public boolean checkParameters() {
return runType != null && (runType.equals(JavaConstants.RUN_TYPE_JAVA) || runType.equals(JavaConstants.RUN_TYPE_JAR) || runType.equals(JavaConstants.RUN_TYPE_NORMAL_JAR)); // Updated condition
}
}
Update the handle method in the JavaTask class to handle the new run type. This may involve adding a new method, such as buildNormalJarCommand, to build the command to execute the NORMAL JAR task:
// Original code
public class JavaTask extends AbstractTask {
// Other code
@Override
public void handle(TaskCallBack taskCallBack) throws TaskException {
// Other code
switch (javaParameters.getRunType()) {
case JavaConstants.RUN_TYPE_JAVA:
command = buildJavaCommand();
break;
case JavaConstants.RUN_TYPE_JAR:
command = buildJarCommand();
break;
default:
throw new RunTypeNotFoundException("run type is required, but it is null now.");
}
// Other code
}
}
// New code
public class JavaTask extends AbstractTask {
// Other code
@Override
public void handle(TaskCallBack taskCallBack) throws TaskException {
// ...
switch (javaParameters.getRunType()) {
case JavaConstants.RUN_TYPE_JAVA:
command = buildJavaCommand();
break;
case JavaConstants.RUN_TYPE_JAR:
command = buildJarCommand();
break;
case JavaConstants.RUN_TYPE_NORMAL_JAR: // New case
command = buildNormalJarCommand(); // New method
break;
default:
throw new RunTypeNotFoundException("run type is required, but it is null now.");
}
// ...
}
// New method
protected String buildNormalJarCommand() {
// Get the absolute path of the JAR file
ResourceContext resourceContext = taskRequest.getResourceContext();
String mainJarAbsolutePathInLocal = resourceContext
.getResourceItem(javaParameters.getMainJar().getResourceName())
.getResourceAbsolutePathInLocal();
// Build the command
StringBuilder builder = new StringBuilder();
builder.append(getJavaCommandPath())
.append("java").append(" ")
.append(buildResourcePath()).append(" ")
.append("-cp").append(" ")
.append(taskRequest.getExecutePath()).append(FOLDER_SEPARATOR)
.append(mainJarAbsolutePathInLocal).append(" ")
.append(javaParameters.getMainClass()).append(" ") // Main class name
.append(javaParameters.getMainArgs().trim()).append(" ") // Main class parameters
.append(javaParameters.getJvmArgs().trim()); // JVM parameters
return builder.toString();
} }
This method first gets the absolute path of the JAR file, then builds a command to execute the NORMAL JAR task. This command includes the path of the Java command, the class path, the path of the JAR file, the main class name, the main class parameters, and the JVM parameters.
@ruanwenjun