hadoop
hadoop copied to clipboard
apache
HDFS-16686. GetJournalEditServlet fails to authorize valid Kerberos request
Description of PR
GetJournalEditServlet uses request.getRemoteuser() to determine the remoteShortName for Kerberos authorization, which fails to match when the JournalNode uses its own Kerberos principal (e.g. jn/
This can be fixed by using the UserGroupInformation provided by the base DfsServlet class using the getUGI(request, conf) call.
How was this patch tested?
Integration tests were performed against an HA configuration running in Kubernetes, running Java 11. With the patch, exceptions which had previously reported expected Kerberos principals which included an IP address string were eliminated.
For code changes:
- [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
- [ ] If applicable, have you updated the
LICENSE,LICENSE-binary,NOTICE-binaryfiles?
This looks like an example of a flaky test, where a previous test execution affects tests that follow. Specifically the reported failed test is testRollback which does not start any JournalNodes, and yet the output includes a waiting JournalNodeSyncer thread. Running testRollback alone never calls the patched servlet GetJournalEditServlet.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 48s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 42m 6s | trunk passed | |
| +1 :green_heart: | compile | 1m 50s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | compile | 1m 42s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 1m 32s | trunk passed | |
| +1 :green_heart: | mvnsite | 1m 44s | trunk passed | |
| +1 :green_heart: | javadoc | 1m 27s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 48s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 57s | trunk passed | |
| +1 :green_heart: | shadedclient | 23m 31s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 1m 21s | the patch passed | |
| +1 :green_heart: | compile | 1m 25s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javac | 1m 25s | the patch passed | |
| +1 :green_heart: | compile | 1m 18s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | javac | 1m 18s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 1m 7s | /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 17 unchanged - 16 fixed = 21 total (was 33) |
| +1 :green_heart: | mvnsite | 1m 27s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 56s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 37s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 41s | the patch passed | |
| +1 :green_heart: | shadedclient | 24m 34s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 247m 11s | hadoop-hdfs in the patch passed. | |
| +1 :green_heart: | asflicense | 1m 7s | The patch does not generate ASF License warnings. | |
| 363m 24s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4724/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4724 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 446430876ee5 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1c8f55262fd86ddd7e4fa79e49d9e488efbdd35d |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4724/5/testReport/ |
| Max. process+thread count | 3293 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4724/5/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 43s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 1s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 1s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 40m 13s | trunk passed | |
| +1 :green_heart: | compile | 1m 43s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | compile | 1m 39s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 1m 24s | trunk passed | |
| +1 :green_heart: | mvnsite | 1m 47s | trunk passed | |
| +1 :green_heart: | javadoc | 1m 25s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 47s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 41s | trunk passed | |
| +1 :green_heart: | shadedclient | 23m 9s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 1m 25s | the patch passed | |
| +1 :green_heart: | compile | 1m 26s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javac | 1m 26s | the patch passed | |
| +1 :green_heart: | compile | 1m 20s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | javac | 1m 20s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 1m 1s | /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 17 unchanged - 16 fixed = 21 total (was 33) |
| +1 :green_heart: | mvnsite | 1m 26s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 58s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 33s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 18s | the patch passed | |
| +1 :green_heart: | shadedclient | 22m 33s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 235m 29s | hadoop-hdfs in the patch passed. | |
| +1 :green_heart: | asflicense | 1m 12s | The patch does not generate ASF License warnings. | |
| 347m 41s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4724/6/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4724 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux fa1e7f3da324 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 3fee70695c9ddea3a896b1f5401abd3fbaeeb5c8 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4724/6/testReport/ |
| Max. process+thread count | 3356 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4724/6/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 43s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 38m 31s | trunk passed | |
| +1 :green_heart: | compile | 1m 44s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | compile | 1m 35s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 1m 24s | trunk passed | |
| +1 :green_heart: | mvnsite | 1m 44s | trunk passed | |
| +1 :green_heart: | javadoc | 1m 25s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 46s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 40s | trunk passed | |
| +1 :green_heart: | shadedclient | 23m 0s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 1m 23s | the patch passed | |
| +1 :green_heart: | compile | 1m 26s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javac | 1m 26s | the patch passed | |
| +1 :green_heart: | compile | 1m 21s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | javac | 1m 21s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 1m 1s | /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 17 unchanged - 16 fixed = 21 total (was 33) |
| +1 :green_heart: | mvnsite | 1m 26s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 59s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 34s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 23s | the patch passed | |
| +1 :green_heart: | shadedclient | 22m 43s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 237m 47s | hadoop-hdfs in the patch passed. | |
| +1 :green_heart: | asflicense | 1m 15s | The patch does not generate ASF License warnings. | |
| 348m 21s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4724/7/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4724 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 1bb5f411c397 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a0e2cac08deed9268a3269b9c2d2e4b3c7a0bd90 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4724/7/testReport/ |
| Max. process+thread count | 3016 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4724/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
