[SPARK-45708][BUILD] Retry mvn deploy

[EnricoMi]

What changes were proposed in this pull request?

Retry uploading artifacts to the Apache repository as often as possible (default is 1).

Why are the changes needed?

It is common to see 408 Request Timeout and 502 Proxy Error when deploying artifacts to Apache snapshot repository: https://github.com/apache/spark/actions/runs/6437635317

2023-10-07T01:09:51.1719360Z [ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:3.0.0-M1:deploy (default-deploy) on project spark-streaming_2.13: ArtifactDeployerException: Failed to deploy artifacts: Could not transfer artifact org.apache.spark:spark-streaming_2.13:jar:tests:3.3.4-20231007.005815-57 from/to apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots): transfer failed for https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-streaming_2.13/3.3.4-SNAPSHOT/spark-streaming_2.13-3.3.4-20231007.005815-57-tests.jar, status: 502 Proxy Error -> [Help 1]

2023-10-07T01:11:48.5651501Z [ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:3.0.0:deploy (default-deploy) on project spark-connect-common_2.12: Failed to deploy artifacts: Could not transfer artifact org.apache.spark:spark-connect-common_2.12:xml:cyclonedx:3.4.2-20231007.001102-103 from/to apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots): transfer failed for https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-connect-common_2.12/3.4.2-SNAPSHOT/spark-connect-common_2.12-3.4.2-20231007.001102-103-cyclonedx.xml, status: 408 Request Timeout -> [Help 1]

Does this PR introduce any user-facing change?

No

How was this patch tested?

Locally: ./buil/mvn deploy:

[INFO] --- deploy:3.1.1:deploy (default-deploy) @ spark-parent_2.13 ---
Downloading from apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/maven-metadata.xml
Downloaded from apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/maven-metadata.xml (1.3 kB at 3.6 kB/s)
Uploading to apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/spark-parent_2.13-4.0.0-20231027.110850-92.pom
Uploading to apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/spark-parent_2.13-4.0.0-20231027.110850-92-tests.jar
Uploading to apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/spark-parent_2.13-4.0.0-20231027.110850-92-cyclonedx.xml
Uploading to apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/spark-parent_2.13-4.0.0-20231027.110850-92-cyclonedx.json
[WARNING] Encountered issue during deployment: Failed to deploy artifacts: Could not transfer artifact org.apache.spark:spark-parent_2.13:pom:4.0.0-20231027.110850-92 from/to apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots): status code: 401, reason phrase: Unauthorized (401)
[INFO] Retrying deployment attempt 2 of 10
...
[INFO] Retrying deployment attempt 10 of 10
Downloading from apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/maven-metadata.xml
Downloaded from apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/maven-metadata.xml (1.3 kB at 7.9 kB/s)
Uploading to apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/spark-parent_2.13-4.0.0-20231027.110850-92.pom
Uploading to apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/spark-parent_2.13-4.0.0-20231027.110850-92-tests.jar
Uploading to apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/spark-parent_2.13-4.0.0-20231027.110850-92-cyclonedx.xml
Uploading to apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/spark-parent_2.13-4.0.0-20231027.110850-92-cyclonedx.json
...
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:3.1.1:deploy (default-deploy) on project spark-parent_2.13: Failed to deploy artifacts: Could not transfer artifact org.apache.spark:spark-parent_2.13:pom:4.0.0-20231027.110850-92 from/to apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots): status code: 401, reason phrase: Unauthorized (401) -> [Help 1]

Was this patch authored or co-authored using generative AI tooling?

No

[EnricoMi]

CC @LuciferYang @HyukjinKwon

[EnricoMi]

@LuciferYang @HyukjinKwon the publish snapshot workflow keeps failing due to HTTP errors: https://github.com/apache/spark/actions/workflows/publish_snapshot.yml

Please consider this fix.

[LuciferYang]

Will adding the - Dmaven.resolver.transport=wagon in MAVEN_OPTS have any effect? The default implementation of Maven 3.9 pieces of resolver has changed from wagon to httpclient

[EnricoMi]

Will adding the - Dmaven.resolver.transport=wagon in MAVEN_OPTS have any effect? The default implementation of Maven 3.9 pieces of resolver has changed from wagon to httpclient

Not sure, looks like wagon is an abstraction on top of HTTP, so under the hood HTTP might still experience 408 Request Timeouts: https://maven.apache.org/wagon/index.html

[EnricoMi]

@HyukjinKwon @LuciferYang this is still an issue: https://github.com/apache/spark/actions/runs/7294496840/job/19879448651

[github-actions[bot]]

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!