[Python][C++] support bazel module

[chaokunyang]

Feature Request

Currently we use bazel workspace to manage the c++ build. We need to use bazel module, and it's the way we use nanobind in the future.

Is your feature request related to a problem? Please describe

• First upgrade bazel6 to bazel 8 in #2279
• use bazel module to manage c++ build

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

The biggest challenge may be pyx_library rule used in fury to build cython extention. This extension is from grpc. And grpc disbled bazel module. It seems that bazel module is supported in https://github.com/grpc/grpc/pull/38456. We need to upgrade grpc to that version at least

[HeTPort]

Can i work on this?

[chaokunyang]

@HeTPort Thanks for your interest!

Yes, you're welcome to work on this. This involves:

1. Upgrading bazel from 6 to 8
2. Converting workspace to bazel module
3. Upgrading gRPC to a version that supports bazel module, or even better, we can remvoe grpc dependency

The main challenge is handling the pyx_library rule for Cython extensions.

Feel free to start with a draft PR. I'm happy to guide you through the process.

Let me know if you have any questions!

[HeTPort]

Hi @chaokunyang, While handling this issue, I found that the bazel file seems not to be complete enough. And upgrading gRPC is necessary but not sufficient. There are a few possible ways:

1. Enable bzlmod (update Bazel to a version with bzlmod) and keep WORKSPACE temporarily.
2. Create MODULE.bazel and bring core deps in via bzlmod
3. Handle Python toolchain and Cython macro: Keep using WORKSPACE for python_configure and pyarrow_configure during the first stage of migration . Or vendor the Cython macro: copy cython_library.bzl into bazel/ with cc_kwargs support and change: load("@com_github_grpc_grpc//bazel:cython_library.bzl", "pyx_library") to load("@fory//bazel:cython_library.bzl", "pyx_library")
4. Convert python_configure and pyarrow_configure to module extensions. Remove WORKSPACE when all repos come from modules/extensions.

From a long-term perspective, which way matches best, or are there any problems with these solutions? Would appreciate some guidance—thanks!

[chaokunyang]

@HeTPort It would be great if we can go to step 3 and 4 directly. This may take lots of efforts, but it would benefit the project in the long run.