dolphinscheduler icon indicating copy to clipboard operation
dolphinscheduler copied to clipboard

[DSIP-27] Deprecate write java code in JAVA task

Open ruanwenjun opened this issue 10 months ago • 7 comments

Search before asking

  • [X] I had searched in the issues and found no similar feature requirement.

Motivation

Right now, we support two kind of usage of JAVA tasks,

image

The first type is JAVA, user can write java code, once the worker receive this kind of task, it will compile the java code to class, and then execute the class file.

The second type is JAR, user can submit a fat jar, once the worker receive this kind of task, it will directly execute the fat jar.

This issue is want to remove the first type of JAVA task, since this is more like a demo, no one will submit java code, and in most of the time, the java application is not written within one file.

So we only need to support submit Jar.

Design Detail

Further more, we would better support user submit jars rather than a fat jar. The command may look like below, this should be more usefully

java $JAVA_OPTS \
  -Dxx.yy.args=1 \
  -cp "xx.jar":"xx.conf" \
  xx.Application args

So we need to change the current task mode into: Fat JAR, NORMAL JAR

Compatibility, Deprecation, and Migration Plan

This is not compatible with the current version.

Test Plan

Add unit test.

Are you willing to submit a PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

ruanwenjun avatar Apr 09 '24 09:04 ruanwenjun

LGTM

xinxingi avatar Apr 23 '24 06:04 xinxingi

The DSIP has been added to OSPP, no further development needs to be specified.

pinkfloyds avatar May 10 '24 11:05 pinkfloyds

How about this

  1. Remove the selection boxes for Run Type, and add dependency package parameters. If this parameter is empty, it means fat jar, and if it is not empty, it means normal jar.
  2. Add Main class parameters.

pinkfloyds avatar May 11 '24 00:05 pinkfloyds

The DSIP has been added to OSPP, no further development needs to be specified.

Why can this be added to OSPP so casually? OSPP has a very long end time, which will block releases.

ruanwenjun avatar May 11 '24 02:05 ruanwenjun

The DSIP has been added to OSPP, no further development needs to be specified.

Why can this be added to OSPP so casually? OSPP has a very long end time, which will block releases.

I apologize for not explaining in advance. Initially, I thought it wasn't a highly important feature, just a few versions of delay.

pinkfloyds avatar May 11 '24 10:05 pinkfloyds

LGTM, In addition, We should be cautious about adding new tasks in the future, and try to receive enough votes and requests to add new tasks.

zhuxt2015 avatar Jun 26 '24 09:06 zhuxt2015

Please assign to me

zhuxt2015 avatar Jun 26 '24 09:06 zhuxt2015

I am a student selected in this OSPP, and here is part of my implementation plan.

1. Delete the Java type

If you want to add a new task type, you need to first delete the related code of the Java type from the previous task types. image

For example, this piece of code first determines whether it is a Java type. You can use this as a starting point to delete the JAVA type and leave a mark. Later, you can add the Normal Jar type. By the same token, in all related places, first delete the code of the Java type task, including calls, uses, etc. By the way, record the position. After deletion is complete and other functions are normal, add the Normal jar type at these recorded places.

2. Partial implementation of the Normal Jar type

First, we need to add a new constant in the JavaConstants class to represent the NORMAL JAR task type:

// Original code
public class JavaConstants {
    public static final String RUN_TYPE_JAVA = "JAVA";
    public static final String RUN_TYPE_JAR = "JAR";
}

// New code
public class JavaConstants {
    public static final String RUN_TYPE_JAVA = "JAVA";
    public static final String RUN_TYPE_JAR = "JAR";
    public static final String RUN_TYPE_NORMAL_JAR = "NORMAL_JAR";  // New constant
}

Then, update the checkParameters method in the JavaParameters class to accept the new run type:

// Original code
public class JavaParameters extends AbstractParameters {
    // ...
    public boolean checkParameters() {
        return runType != null && (runType.equals(JavaConstants.RUN_TYPE_JAVA) || runType.equals(JavaConstants.RUN_TYPE_JAR));
    }
}

// New code
public class JavaParameters extends AbstractParameters {
    // ...
    public boolean checkParameters() {
        return runType != null && (runType.equals(JavaConstants.RUN_TYPE_JAVA) || runType.equals(JavaConstants.RUN_TYPE_JAR) || runType.equals(JavaConstants.RUN_TYPE_NORMAL_JAR));  // Updated condition
    }
}

Update the handle method in the JavaTask class to handle the new run type. This may involve adding a new method, such as buildNormalJarCommand, to build the command to execute the NORMAL JAR task:

// Original code
public class JavaTask extends AbstractTask {
    // Other code
    @Override
    public void handle(TaskCallBack taskCallBack) throws TaskException {
        // Other code
        switch (javaParameters.getRunType()) {
            case JavaConstants.RUN_TYPE_JAVA:
                command = buildJavaCommand();
                break;
            case JavaConstants.RUN_TYPE_JAR:
                command = buildJarCommand();
                break;
            default:
                throw new RunTypeNotFoundException("run type is required, but it is null now.");
        }
        // Other code
    }
}

// New code
public class JavaTask extends AbstractTask {
    // Other code
    @Override
    public void handle(TaskCallBack taskCallBack) throws TaskException {
        // ...
        switch (javaParameters.getRunType()) {
            case JavaConstants.RUN_TYPE_JAVA:
                command = buildJavaCommand();
                break;
            case JavaConstants.RUN_TYPE_JAR:
                command = buildJarCommand();
                break;
            case JavaConstants.RUN_TYPE_NORMAL_JAR:  // New case
                command = buildNormalJarCommand();  // New method
                break;
            default:
                throw new RunTypeNotFoundException("run type is required, but it is null now.");
        }
        // ...
    }

    // New method
   protected String buildNormalJarCommand() {
    // Get the absolute path of the JAR file
    ResourceContext resourceContext = taskRequest.getResourceContext();
    String mainJarAbsolutePathInLocal = resourceContext
            .getResourceItem(javaParameters.getMainJar().getResourceName())
            .getResourceAbsolutePathInLocal();

    // Build the command
    StringBuilder builder = new StringBuilder();
    builder.append(getJavaCommandPath())
           .append("java").append(" ")
           .append(buildResourcePath()).append(" ")
           .append("-cp").append(" ")
           .append(taskRequest.getExecutePath()).append(FOLDER_SEPARATOR)
           .append(mainJarAbsolutePathInLocal).append(" ")
           .append(javaParameters.getMainClass()).append(" ")  // Main class name
           .append(javaParameters.getMainArgs().trim()).append(" ")  // Main class parameters
           .append(javaParameters.getJvmArgs().trim());  // JVM parameters

    return builder.toString();
} }

This method first gets the absolute path of the JAR file, then builds a command to execute the NORMAL JAR task. This command includes the path of the Java command, the class path, the path of the JAR file, the main class name, the main class parameters, and the JVM parameters.

ailiujiarui avatar Jul 10 '24 08:07 ailiujiarui

@ruanwenjun

ailiujiarui avatar Jul 22 '24 04:07 ailiujiarui