HADOOP-19024. Use bouncycastle jdk18 1.77

[pjfanning]

Description of PR

HADOOP-19024

bouncycastle jdk15 jars are no longer patched. Security fixes only go on the jdk18 jars.

How was this patch tested?

CI build

For code changes:

• [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
• [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
• [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• [x] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

[steveloughran]

I agree that this is needed

[hadoop-yetus]

:broken_heart: -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 49s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 0s		No case conflicting files found.
+0 :ok:	codespell	0m 1s		codespell was not available.
+0 :ok:	detsecrets	0m 1s		detect-secrets was not available.
+0 :ok:	xmllint	0m 1s		xmllint was not available.
+0 :ok:	markdownlint	0m 1s		markdownlint was not available.
+0 :ok:	shelldocs	0m 1s		Shelldocs was not available.
+1 :green_heart:	@author	0m 0s		The patch does not contain any @author tags.
-1 :x:	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 :ok:	mvndep	14m 49s		Maven dependency ordering for branch
+1 :green_heart:	mvninstall	35m 34s		trunk passed
+1 :green_heart:	compile	18m 34s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart:	compile	16m 43s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart:	mvnsite	19m 44s		trunk passed
+1 :green_heart:	javadoc	8m 54s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart:	javadoc	7m 38s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart:	shadedclient	53m 29s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 :ok:	mvndep	0m 38s		Maven dependency ordering for patch
+1 :green_heart:	mvninstall	44m 34s		the patch passed
+1 :green_heart:	compile	17m 46s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart:	javac	17m 46s		the patch passed
+1 :green_heart:	compile	16m 37s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart:	javac	16m 37s		the patch passed
+1 :green_heart:	blanks	0m 0s		The patch has no blanks issues.
+1 :green_heart:	mvnsite	15m 45s		the patch passed
+1 :green_heart:	shellcheck	0m 0s		No new issues.
+1 :green_heart:	javadoc	8m 52s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart:	javadoc	7m 32s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart:	shadedclient	54m 47s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 :x:	unit	781m 59s	/patch-unit-root.txt	root in the patch passed.
+1 :green_heart:	asflicense	1m 29s		The patch does not generate ASF License warnings.
		1098m 59s

Reason	Tests
Failed junit tests	hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2
	hadoop.hdfs.server.datanode.TestDirectoryScanner

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6410/1/artifact/out/Dockerfile
GITHUB PR	https://github.com/apache/hadoop/pull/6410
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint markdownlint shellcheck shelldocs
uname	Linux f3e66b6999b8 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / c5d946ceb3e2254486666ce9932002da794a0127
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6410/1/testReport/
Max. process+thread count	2517 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs-nfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-tools/hadoop-streaming hadoop-tools/hadoop-gridmix hadoop-tools/hadoop-extras hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure hadoop-client-modules/hadoop-client-integration-tests hadoop-cloud-storage-project/hadoop-cos . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6410/1/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[slfan1989]

@pjfanning Thank you for your contribution! LGTM.

[slfan1989]

@pjfanning Can we retrigger compilation?

[hadoop-yetus]

:broken_heart: -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 34s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 0s		No case conflicting files found.
+0 :ok:	codespell	0m 1s		codespell was not available.
+0 :ok:	detsecrets	0m 1s		detect-secrets was not available.
+0 :ok:	xmllint	0m 1s		xmllint was not available.
+0 :ok:	markdownlint	0m 1s		markdownlint was not available.
+0 :ok:	shelldocs	0m 1s		Shelldocs was not available.
+1 :green_heart:	@author	0m 0s		The patch does not contain any @author tags.
-1 :x:	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 :ok:	mvndep	14m 15s		Maven dependency ordering for branch
+1 :green_heart:	mvninstall	30m 44s		trunk passed
+1 :green_heart:	compile	16m 28s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart:	compile	14m 58s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart:	mvnsite	18m 56s		trunk passed
+1 :green_heart:	javadoc	8m 27s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart:	javadoc	7m 32s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart:	shadedclient	47m 16s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 :ok:	mvndep	0m 39s		Maven dependency ordering for patch
+1 :green_heart:	mvninstall	38m 34s		the patch passed
+1 :green_heart:	compile	15m 54s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart:	javac	15m 54s		the patch passed
+1 :green_heart:	compile	14m 50s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart:	javac	14m 50s		the patch passed
+1 :green_heart:	blanks	0m 1s		The patch has no blanks issues.
+1 :green_heart:	mvnsite	13m 41s		the patch passed
+1 :green_heart:	shellcheck	0m 0s		No new issues.
+1 :green_heart:	javadoc	8m 19s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart:	javadoc	7m 37s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart:	shadedclient	49m 1s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 :x:	unit	740m 17s	/patch-unit-root.txt	root in the patch passed.
+1 :green_heart:	asflicense	1m 27s		The patch does not generate ASF License warnings.
		1022m 46s

Reason	Tests
Failed junit tests	hadoop.fs.http.client.TestHttpFSWithHttpFSFileSystem

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6410/2/artifact/out/Dockerfile
GITHUB PR	https://github.com/apache/hadoop/pull/6410
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint markdownlint shellcheck shelldocs
uname	Linux c4321fe5bb8c 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 5ed583ff04076c6992cd56aaf946c05d8e622a68
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6410/2/testReport/
Max. process+thread count	4233 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs-nfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-tools/hadoop-streaming hadoop-tools/hadoop-gridmix hadoop-tools/hadoop-extras hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure hadoop-client-modules/hadoop-client-integration-tests hadoop-cloud-storage-project/hadoop-cos . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6410/2/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[slfan1989]

@pjfanning The changes do have an impact on the unit tests. I will backport this PR to my local environment later and give it a try.

cc @steveloughran @Hexiaoqiao

[slfan1989]

@pjfanning Is this a necessary fix for hadoop-3.4.0? If not, we can fix it in the next version(hadoop-3.4.1).

cc: @Hexiaoqiao

[pjfanning]

It fixes a CVE but it can wait till 3.4.1

[pjfanning]

@slfan1989 is this change something that we can consider to merge to trunk and possibly branch-3.4?