[Bug] [WORKER-SERVER] Failed to start in 3.3.0-alpha with helm installation

[e7868a]

Search before asking

• [x] I had searched in the issues and found no similar issues.

What happened

install dolphinscheduler-3.3.0-alpha with helm:

helm install dolphin2 dolphinscheduler-helm-3.3.0-alpha.tgz

worker-server failed to start,with error:

Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'org.apache.dolphinscheduler.plugin.storage.api.StorageOperator' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {}
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.raiseNoMatchingBeanFound(DefaultListableBeanFactory.java:1801)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1396)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311)
        at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887)
        at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791)

What you expected to happen

worker-server should start.

How to reproduce

··· helm install dolphin2 dolphinscheduler-helm-3.3.0-alpha.tgz ···

Anything else

No response

Version

3.3.0-alpha

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [x] I agree to follow this project's Code of Conduct

[SbloodyS]

Helm deployment is not perfect at present. You can try using another mode instead.

[e7868a]

I discovered this issue due to the incorrect mounting path configuration for common.properties in the Helm chart. In previous versions, common.properties was mounted at /opt/dolphinscheduler/conf/common.properties. However, in version 3.3.0, it needs to be mounted in each module's own conf directory, for example:

/opt/dolphinscheduler/master-server/conf/common.properties
/opt/dolphinscheduler/worker-server/conf/common.properties
/opt/dolphinscheduler/api-server/conf/common.properties
/opt/dolphinscheduler/alert-server/conf/common.properties

Additionally, since the Helm deployment uses S3 storage by default, we also need to add dolphinscheduler-storage-s3-3.3.0-alpha.jar to the class path.

According to the current version's image implementation, we need to:

1. Create the /opt/dolphinscheduler/plugins/storage-plugins directory
2. Either copy the /opt/dolphinscheduler/libs/dolphinscheduler-storage-s3-3.3.0-alpha.jar into this directory, or create a symbolic link to it.

[DhiyaneshwaranR]

Hi @SbloodyS I am willing to work on this.

[hiql]

In version 3.3.0, there is no plugin packaged by default，you have to copy them by yourself, or run maven with -Dbuild.plugins.skip=false

[SbloodyS]

Hi @SbloodyS I am willing to work on this.

Done.

[liukai237]

How to add dolphinscheduler-storage-s3-3.3.0-alpha.jar into IntelliJ IDEA?

[anjia0532]

save as ds_fix.sh

#!/bin/bash
set -eo pipefail

MODULES_PATH=(
alert-server
api-server
master-server
tools
worker-server
)

DOLPHINSCHEDULER_HOME=${DOLPHINSCHEDULER_HOME:-/opt/dolphinscheduler/}

for MODULE in ${MODULES_PATH[@]}; do
    MODULE_LIB_DIR="${DOLPHINSCHEDULER_HOME}/${MODULE}/libs"
    ls -lah $MODULE_LIB_DIR | grep dolphinscheduler-storage
    for jar in $(find $MODULE_LIB_DIR/* -name "dolphinscheduler-storage*.jar" -execdir echo {} ';'); do
        unlink $MODULE_LIB_DIR/$jar
        echo "unlink $MODULE_LIB_DIR/$jar"
    done

    for jar in $(find $DOLPHINSCHEDULER_HOME/libs/* -name "dolphinscheduler-storage*.jar" -execdir echo {} ';'); do
        ln -s $DOLPHINSCHEDULER_HOME/libs/$jar $MODULE_LIB_DIR/$jar
        echo "ln -s $DOLPHINSCHEDULER_HOME/libs/$jar $MODULE_LIB_DIR/$jar"
    done
    ls -lah $MODULE_LIB_DIR | grep dolphinscheduler-storage

    for cfg in $(ls $DOLPHINSCHEDULER_HOME/conf/ | grep "properties\|yaml"); do
        cp $DOLPHINSCHEDULER_HOME/conf/$cfg ${DOLPHINSCHEDULER_HOME}/${MODULE}/conf/
        echo "cp $DOLPHINSCHEDULER_HOME/conf/$cfg ${DOLPHINSCHEDULER_HOME}/${MODULE}/conf/"
    done
done

bash ${DOLPHINSCHEDULER_HOME}/$1/bin/start.sh

chmod +x ds_fix.sh

# run  worker module
./ds_fix.sh worker_server

# run master module
./ds_fix.sh  master-server

# ignore other module e.g. tools,api,alert
``

[anjia0532]

1. create .mvn/wrapper/maven-wrapper.properties
2. install-plugins.sh to download plugins by plugins_config
3. run start.sh