migrate from Apache HttpClient 4 to Apache HttpClient 5

[sullis]

https://hc.apache.org/httpcomponents-client-5.3.x/migration-guide/index.html

https://hc.apache.org/httpcomponents-client-5.3.x/migration-guide/migration-to-classic.html

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 23.23651% with 185 lines in your changes missing coverage. Please review.

Project coverage is 62.11%. Comparing base (59551e4) to head (32f64e4). Report is 715 commits behind head on master.

Files	Patch %	Lines
...e/pinot/common/utils/FileUploadDownloadClient.java	10.00%	36 Missing :warning:
...org/apache/pinot/common/utils/http/HttpClient.java	23.40%	36 Missing :warning:
...pinot/common/utils/fetcher/HttpSegmentFetcher.java	0.00%	26 Missing :warning:
...oller/api/resources/PinotRunningQueryResource.java	0.00%	18 Missing :warning:
...he/pinot/common/utils/webhdfs/WebHdfsV1Client.java	0.00%	17 Missing :warning:
...a/org/apache/pinot/common/minion/MinionClient.java	42.85%	13 Missing and 3 partials :warning:
...ontroller/api/resources/PinotControllerLogger.java	0.00%	11 Missing :warning:
...ava/org/apache/pinot/common/utils/SchemaUtils.java	0.00%	8 Missing :warning:
...pache/pinot/connector/spark/common/HttpUtils.scala	0.00%	8 Missing :warning:
...inot/plugin/provider/AzureEnvironmentProvider.java	46.15%	7 Missing :warning:
... and 2 more

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #13222      +/-   ##
============================================
+ Coverage     61.75%   62.11%   +0.35%     
+ Complexity      207      198       -9     
============================================
  Files          2436     2557     +121     
  Lines        133233   140890    +7657     
  Branches      20636    21860    +1224     
============================================
+ Hits          82274    87508    +5234     
- Misses        44911    46765    +1854     
- Partials       6048     6617     +569     

Flag	Coverage Δ
custom-integration1	<0.01% <0.00%> (-0.01%)	:arrow_down:
integration	<0.01% <0.00%> (-0.01%)	:arrow_down:
integration1	<0.01% <0.00%> (-0.01%)	:arrow_down:
integration2	0.00% <0.00%> (ø)
java-11	62.05% <23.23%> (+0.34%)	:arrow_up:
java-21	61.99% <23.23%> (+0.37%)	:arrow_up:
skip-bytebuffers-false	62.07% <23.23%> (+0.32%)	:arrow_up:
skip-bytebuffers-true	61.97% <23.23%> (+34.24%)	:arrow_up:
temurin	62.11% <23.23%> (+0.35%)	:arrow_up:
unittests	62.10% <23.23%> (+0.35%)	:arrow_up:
unittests1	46.69% <22.22%> (-0.20%)	:arrow_down:
unittests2	27.65% <6.63%> (-0.08%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[sullis]

ready for review @Jackie-Jiang

[sullis]

GitHub Actions: All checks have passed.

[sullis]

Rebased.

[sullis]

Rebased.

[sullis]

Thanks for taking time to do this upgrade! Do we still need to keep httpclient 4 related dependencies?

For now, I think we need to keep httpclient 4 in the project's root pom.xml

I tried removing HttpClient4 and it caused dependency conflicts. Apache HttpClient 4 is a transitive dependency across multiple libraries: AWS SDK 2.x Apache Client, google-api-client, and helix-core