dolphinscheduler
dolphinscheduler copied to clipboard
apache
[Bug] [Deploy]The modified common.properties didn't work in k8s dockerfile with copy command!
Search before asking
- [X] I had searched in the issues and found no similar issues.
What happened
After deployed the dolphhinscheduler by k8s, I wanted to create a tenant, but it happend '创建租户错误'
From then pod,I got the log
[INFO] 2023-06-07 17:55:58.708 +0800 org.apache.dolphinscheduler.api.aspect.AccessLogAspect:[91] - REQUEST TRACE_ID:7e600b79-4d4f-4b49-9a1d-f188226fd135, LOGIN_USER:admin, URI:/dolphinscheduler/tenants/verify-code, METHOD:GET, HANDLER:org.apache.dolphinscheduler.api.controller.TenantController.verifyTenantCode, ARGS:{tenantCode=ddd}
[INFO] 2023-06-07 17:55:59.019 +0800 org.apache.dolphinscheduler.api.aspect.AccessLogAspect:[91] - REQUEST TRACE_ID:7e5884fb-bfff-43e1-b68a-3d4a7db708b3, LOGIN_USER:admin, URI:/dolphinscheduler/tenants, METHOD:POST, HANDLER:org.apache.dolphinscheduler.api.controller.TenantController.createTenant, ARGS:{queueId=1, description=, tenantCode=ddd}
[ERROR] 2023-06-07 17:55:59.027 +0800 org.apache.dolphinscheduler.api.exceptions.ApiExceptionHandler:[53] - 创建租户错误
java.lang.NullPointerException: null
at org.apache.dolphinscheduler.service.storage.impl.HadoopUtils.mkdir(HadoopUtils.java:293)
at org.apache.dolphinscheduler.service.storage.impl.HadoopUtils.createTenantDirIfNotExists(HadoopUtils.java:268)
at org.apache.dolphinscheduler.api.service.impl.TenantServiceImpl.createTenant(TenantServiceImpl.java:160)
at org.apache.dolphinscheduler.api.service.impl.TenantServiceImpl$$FastClassBySpringCGLIB$$fcc8d845.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
maybe it depended on hadoop, then I read the source code
I changes the common.properties from resource.storage.type=HDFS to resource.storage.type=NONE ,and then built then image by COPY common.properties /opt/dolphinscheduler/conf in dockerfile. but it did'nt work.
In then pod ,the file didn't change
I had tried a lot !!!
What you expected to happen
the file common.properties changed and created tenants successfully!
How to reproduce
deploy by k8s ,then create tenant
Anything else
No response
Version
3.1.x
Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
Search before asking
- [X] I had searched in the issues and found no similar issues.
What happened
After deployed the dolphhinscheduler by k8s, I wanted to create a tenant, but it happend '创建租户错误'
From then pod,I got the log
[INFO] 2023-06-07 17:55:58.708 +0800 org.apache.dolphinscheduler.api.aspect.AccessLogAspect:[91] - REQUEST TRACE_ID:7e600b79-4d4f-4b49-9a1d-f188226fd135, LOGIN_USER:admin, URI:/dolphinscheduler/tenants/verify-code, METHOD:GET, HANDLER:org.apache.dolphinscheduler.api.controller.TenantController.verifyTenantCode, ARGS:{tenantCode=ddd}
[INFO] 2023-06-07 17:55:59.019 +0800 org.apache.dolphinscheduler.api.aspect.AccessLogAspect:[91] - REQUEST TRACE_ID:7e5884fb-bfff-43e1-b68a-3d4a7db708b3, LOGIN_USER:admin, URI:/dolphinscheduler/tenants, METHOD:POST, HANDLER:org.apache.dolphinscheduler.api.controller.TenantController.createTenant, ARGS:{queueId=1, description=, tenantCode=ddd}
[ERROR] 2023-06-07 17:55:59.027 +0800 org.apache.dolphinscheduler.api.exceptions.ApiExceptionHandler:[53] - 创建租户错误
java.lang.NullPointerException: null
at org.apache.dolphinscheduler.service.storage.impl.HadoopUtils.mkdir(HadoopUtils.java:293)
at org.apache.dolphinscheduler.service.storage.impl.HadoopUtils.createTenantDirIfNotExists(HadoopUtils.java:268)
at org.apache.dolphinscheduler.api.service.impl.TenantServiceImpl.createTenant(TenantServiceImpl.java:160)
at org.apache.dolphinscheduler.api.service.impl.TenantServiceImpl$$FastClassBySpringCGLIB$$fcc8d845.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
maybe it depended on hadoop, then I read the source code
I changes the common.properties from resource.storage.type=HDFS to resource.storage.type=NONE ,and then built then image by COPY common.properties /opt/dolphinscheduler/conf in dockerfile. but it did'nt work.
In then pod ,the file didn't change
I had tried a lot !!!
What you expected to happen
the file common.properties changed and created tenants successfully!
How to reproduce
deploy by k8s ,then create tenant
Anything else
No response
Version
3.1.x
Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
- In order for us to understand your request as soon as possible, please provide detailed information, version or pictures.
- If you haven't received a reply for a long time, you can join our slack and send your question to channel
#troubleshooting
Hi, @XQFHNB Whether files can be copied into the image should be a problem limited to docker, and has nothing to do with ds itself. Could you please provide more information on how you built the image.
I found the point, changed vaules.yaml from resource.storage.type=HDFS to resource.storage.type=NONE
我找到了重点，将 vaules.yaml 从 更改
resource.storage.type=HDFS为resource.storage.type=NONE
Did you solve the problem, I solved the same problem：
[ERROR] 2023-07-10 12:11:46.810 +0800 org.apache.dolphinscheduler.api.exceptions.ApiExceptionHandler:[53] - 创建租户错误
java.lang.NullPointerException: null
at org.apache.dolphinscheduler.service.storage.impl.HadoopUtils.mkdir(HadoopUtils.java:293)
at org.apache.dolphinscheduler.service.storage.impl.HadoopUtils.createTenantDirIfNotExists(HadoopUtils.java:268)
at org.apache.dolphinscheduler.api.service.impl.TenantServiceImpl.createTenant(TenantServiceImpl.java:160)
at org.apache.dolphinscheduler.api.service.impl.TenantServiceImpl$$FastClassBySpringCGLIB$$fcc8d845.invoke(
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
需要将value.yaml中的配置修改为hdfs common:
Configmap
configmap: RESOURCE_STORAGE_TYPE: "HDFS" FS_DEFAULT_FS: "file:///" DOLPHINSCHEDULER_OPTS: "" DATA_BASEDIR_PATH: "/tmp/dolphinscheduler" RESOURCE_UPLOAD_PATH: "/dolphinscheduler"
resource storage type: HDFS, S3, OSS, GCS, ABS, NONE
resource.storage.type: HDFS
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.