tez icon indicating copy to clipboard operation
tez copied to clipboard

fs.getFukeStatus NullPointException deal

Open chongchuanbing opened this issue 4 years ago • 1 comments

when the path is root and it has the excute permisson, it will continue get parent path to judge permisson. but the parent path of root will cause the NullPointException.

the statck trace:

2020-05-15T14:57:17,544 ERROR [HiveServer2-Background-Pool: Thread-117] exec.Task: Failed to execute tez graph.
org.apache.hadoop.ipc.RemoteException: java.lang.NullPointerException

        at org.apache.hadoop.ipc.Client.call(Client.java:1476) ~[hadoop-common-2.7.7.jar:?]
        at org.apache.hadoop.ipc.Client.call(Client.java:1413) ~[hadoop-common-2.7.7.jar:?]
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) ~[hadoop-common-2.7.7.jar:?]
        at com.sun.proxy.$Proxy30.getFileInfo(Unknown Source) ~[?:?]
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:776) ~[hadoop-hdfs-2.7.7.jar:?]
        at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_221]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_221]
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) ~[hadoop-common-2.7.7.jar:?]
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) ~[hadoop-common-2.7.7.jar:?]
        at com.sun.proxy.$Proxy31.getFileInfo(Unknown Source) ~[?:?]
        at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2117) ~[hadoop-hdfs-2.7.7.jar:?]
        at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305) ~[hadoop-hdfs-2.7.7.jar:?]
        at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301) ~[hadoop-hdfs-2.7.7.jar:?]
        at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) ~[hadoop-common-2.7.7.jar:?]
        at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301) ~[hadoop-hdfs-2.7.7.jar:?]
        at org.apache.tez.client.TezClientUtils.checkAncestorPermissionsForAllUsers(TezClientUtils.java:1033) ~[tez-api-0.9.2.jar:0.9.2]
        at org.apache.tez.client.TezClientUtils.addLocalResources(TezClientUtils.java:277) ~[tez-api-0.9.2.jar:0.9.2]
        at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:185) ~[tez-api-0.9.2.jar:0.9.2]
        at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:1156) ~[tez-api-0.9.2.jar:0.9.2]
        at org.apache.tez.client.TezClient.setupApplicationContext(TezClient.java:473) ~[tez-api-0.9.2.jar:0.9.2]
        at org.apache.tez.client.TezClient.start(TezClient.java:401) ~[tez-api-0.9.2.jar:0.9.2]
        at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:376) ~[hive-exec-2.3.6.jar:2.3.6]
        at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:323) ~[hive-exec-2.3.6.jar:2.3.6]
        at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager$TezSessionPoolSession.openInternal(TezSessionPoolManager.java:703) ~[hive-exec-2.3.6.jar:2.3.6]
        at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:196) ~[hive-exec-2.3.6.jar:2.3.6]
        at org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:303) ~[hive-exec-2.3.6.jar:2.3.6]
        at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:168) ~[hive-exec-2.3.6.jar:2.3.6]
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) ~[hive-exec-2.3.6.jar:2.3.6]

chongchuanbing avatar May 18 '20 05:05 chongchuanbing

:broken_heart: -1 overall

Vote Subsystem Runtime Comment
+0 :ok: reexec 1m 17s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+1 :green_heart: mvninstall 13m 55s master passed
+1 :green_heart: compile 0m 34s master passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 :green_heart: compile 0m 30s master passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: checkstyle 0m 55s master passed
+1 :green_heart: javadoc 0m 44s master passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 :green_heart: javadoc 0m 35s master passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+0 :ok: spotbugs 1m 32s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 :green_heart: findbugs 1m 29s master passed
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 24s the patch passed
+1 :green_heart: compile 0m 25s the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 :green_heart: javac 0m 25s the patch passed
+1 :green_heart: compile 0m 21s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: javac 0m 21s the patch passed
+1 :green_heart: checkstyle 0m 13s the patch passed
+1 :green_heart: whitespace 0m 0s The patch has no whitespace issues.
+1 :green_heart: javadoc 0m 25s the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 :green_heart: javadoc 0m 25s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: findbugs 1m 6s the patch passed
_ Other Tests _
+1 :green_heart: unit 2m 5s tez-api in the patch passed.
+1 :green_heart: asflicense 0m 13s The patch does not generate ASF License warnings.
26m 23s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-67/1/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/tez/pull/67
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname Linux 9d1ef122daea 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/tez.sh
git revision master / 6d7ef203c
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-67/1/testReport/
Max. process+thread count 252 (vs. ulimit of 5500)
modules C: tez-api U: tez-api
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-67/1/console
versions git=2.25.1 maven=3.6.3 findbugs=3.0.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

tez-yetus avatar Feb 01 '22 04:02 tez-yetus