hadoop icon indicating copy to clipboard operation
hadoop copied to clipboard

HADOOP-18235. vulnerability: we may leak sensitive information in LocalKeyStoreProvider

Open cbaenziger opened this issue 3 years ago • 2 comments

Description of PR

It is to ensure we have a file and have set permissions on the file before writing out data. I simply worked to rearrange the current logic and was unaware if there may be a better pattern to follow else where in Hadoop.

How was this patch tested?

This is an untested PR. I have merely verified it builds.

For code changes:

  • [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • [N/A] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • [N/A] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • [N/A] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

cbaenziger avatar Oct 10 '22 18:10 cbaenziger

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 45s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 39m 24s trunk passed
+1 :green_heart: compile 23m 22s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 :green_heart: compile 20m 51s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 :green_heart: checkstyle 1m 29s trunk passed
+1 :green_heart: mvnsite 1m 55s trunk passed
+1 :green_heart: javadoc 1m 31s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 1m 8s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 2m 56s trunk passed
+1 :green_heart: shadedclient 23m 37s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 1m 4s the patch passed
+1 :green_heart: compile 22m 47s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javac 22m 47s the patch passed
+1 :green_heart: compile 20m 58s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 :green_heart: javac 20m 58s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 1m 30s the patch passed
+1 :green_heart: mvnsite 2m 0s the patch passed
+1 :green_heart: javadoc 1m 22s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 1m 2s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
-1 :x: spotbugs 2m 54s /new-spotbugs-hadoop-common-project_hadoop-common.html hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 :green_heart: shadedclient 23m 20s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 18m 58s hadoop-common in the patch passed.
+1 :green_heart: asflicense 1m 15s The patch does not generate ASF License warnings.
215m 22s
Reason Tests
SpotBugs module:hadoop-common-project/hadoop-common
Exceptional return value of java.io.File.createNewFile() ignored in org.apache.hadoop.security.alias.LocalKeyStoreProvider.flush() At LocalKeyStoreProvider.java:ignored in org.apache.hadoop.security.alias.LocalKeyStoreProvider.flush() At LocalKeyStoreProvider.java:[line 147]
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4998/1/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/4998
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 44ecefdc1706 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / b408f76d010b613e58b4f68d33c4e6d3149dc78f
Default Java Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4998/1/testReport/
Max. process+thread count 3159 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4998/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Oct 10 '22 22:10 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 1m 2s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 42m 8s trunk passed
+1 :green_heart: compile 25m 35s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 :green_heart: compile 22m 1s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 :green_heart: checkstyle 1m 26s trunk passed
+1 :green_heart: mvnsite 1m 54s trunk passed
+1 :green_heart: javadoc 1m 25s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 0m 57s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 3m 0s trunk passed
+1 :green_heart: shadedclient 25m 53s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 1m 6s the patch passed
+1 :green_heart: compile 24m 51s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javac 24m 51s the patch passed
+1 :green_heart: compile 22m 6s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 :green_heart: javac 22m 6s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 1m 18s the patch passed
+1 :green_heart: mvnsite 1m 52s the patch passed
+1 :green_heart: javadoc 1m 16s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 0m 57s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
-1 :x: spotbugs 3m 1s /new-spotbugs-hadoop-common-project_hadoop-common.html hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 :green_heart: shadedclient 26m 20s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 18m 34s hadoop-common in the patch passed.
+1 :green_heart: asflicense 1m 10s The patch does not generate ASF License warnings.
228m 23s
Reason Tests
SpotBugs module:hadoop-common-project/hadoop-common
Exceptional return value of java.io.File.createNewFile() ignored in org.apache.hadoop.security.alias.LocalKeyStoreProvider.flush() At LocalKeyStoreProvider.java:ignored in org.apache.hadoop.security.alias.LocalKeyStoreProvider.flush() At LocalKeyStoreProvider.java:[line 147]
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4998/2/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/4998
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 9019c7bbd85e 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 5d045909b32ff03a576e18822b4235a5c6dc07bf
Default Java Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4998/2/testReport/
Max. process+thread count 3137 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4998/2/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Oct 11 '22 20:10 hadoop-yetus

where are we with this patch? can/should we get it into 3.3.5

steveloughran avatar Jan 18 '23 14:01 steveloughran

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 6m 42s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 34m 55s trunk passed
+1 :green_heart: compile 9m 1s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: compile 8m 17s trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 :green_heart: checkstyle 0m 43s trunk passed
+1 :green_heart: mvnsite 0m 56s trunk passed
+1 :green_heart: javadoc 0m 45s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javadoc 0m 36s trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 :green_heart: spotbugs 1m 30s trunk passed
+1 :green_heart: shadedclient 21m 16s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 32s the patch passed
+1 :green_heart: compile 8m 33s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javac 8m 33s the patch passed
+1 :green_heart: compile 8m 9s the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 :green_heart: javac 8m 9s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 39s the patch passed
+1 :green_heart: mvnsite 0m 55s the patch passed
+1 :green_heart: javadoc 0m 43s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javadoc 0m 36s the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
-1 :x: spotbugs 1m 40s /new-spotbugs-hadoop-common-project_hadoop-common.html hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 :green_heart: shadedclient 21m 19s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 16m 3s hadoop-common in the patch passed.
+1 :green_heart: asflicense 0m 43s The patch does not generate ASF License warnings.
146m 57s
Reason Tests
SpotBugs module:hadoop-common-project/hadoop-common
Exceptional return value of java.io.File.createNewFile() ignored in org.apache.hadoop.security.alias.LocalKeyStoreProvider.flush() At LocalKeyStoreProvider.java:ignored in org.apache.hadoop.security.alias.LocalKeyStoreProvider.flush() At LocalKeyStoreProvider.java:[line 147]
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4998/1/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/4998
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 8a78c1cdd11e 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 5d045909b32ff03a576e18822b4235a5c6dc07bf
Default Java Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4998/1/testReport/
Max. process+thread count 3020 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4998/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Apr 11 '24 21:04 hadoop-yetus