spark icon indicating copy to clipboard operation
spark copied to clipboard

[SPARK-46919][BUILD][CONNECT] Upgrade `grpcio*` to 1.60.0 and `grpc-java` to 1.61.0

Open LuciferYang opened this issue 1 year ago • 1 comments

What changes were proposed in this pull request?

This PR aims to upgrade grpcio* from 1.59.3 to 1.60.1and grpc-java from 1.59.0 to 1.61.0 for Apache Spark 4.0.0.

Why are the changes needed?

grpc 1.60.0 start to support dualstack IPv4 and IPv6 backend support:

  • Implemented dualstack IPv4 and IPv6 backend support, as per draft gRFC A61. xDS support currently guarded by GRPC_EXPERIMENTAL_XDS_DUALSTACK_ENDPOINTS env var.

And for Python, there is a new bug fix:

  • [Bug Fix] Change return of grpc.aio.ServicerContext.code() from integer to grpc.StatusCode.

grpc-java 1.61.0 bring some Improvements like:

  • https://github.com/grpc/grpc-java/pull/3279

Note that in grpc-java 1.61.0, since the dependency scope of grpc-protobuf on grpc-protobuf-lite has been changed from compile to runtime, we need to manually configure the dependency of the connect module on grpc-protobuf-lite and explicitly exclude the dependency on protobuf-javalite because SparkConnectService uses io.grpc.protobuf.lite.ProtoLiteUtils

  • https://github.com/grpc/grpc-java/pull/10756/files

The relevant release notes are as follows:

  • https://github.com/grpc/grpc/releases/tag/v1.60.0
  • https://github.com/grpc/grpc/releases/tag/v1.60.1
  • https://github.com/grpc/grpc-java/releases/tag/v1.60.0
  • https://github.com/grpc/grpc-java/releases/tag/v1.60.1
  • https://github.com/grpc/grpc-java/releases/tag/v1.61.0

Does this PR introduce any user-facing change?

No

How was this patch tested?

Pass GitHub Actions

Was this patch authored or co-authored using generative AI tooling?

No

LuciferYang avatar Jan 29 '24 07:01 LuciferYang

1.61.0 released, mark this one to draft first

  • https://github.com/grpc/grpc/releases/tag/v1.61.0

LuciferYang avatar Feb 04 '24 05:02 LuciferYang

The version has been unified to the 1.62 series now

LuciferYang avatar Feb 28 '24 13:02 LuciferYang

Thanks @dongjoon-hyun and @srowen

LuciferYang avatar Mar 13 '24 06:03 LuciferYang