Prepare Python 3.11 support

[csko]

Why are these changes needed?

Preliminary PR for adding Python 3.11 support, mapping out various dependencies, fixing issues.

Checklist:

• cython
  • [x] remove deprecated recursion_depth
  • [x] exc_type https://github.com/cython/cython/issues/4500
• [ ] package dependencies https://pyreadiness.org/3.11/
  • [ ] llvmlite https://github.com/numba/llvmlite/pull/869
  • [ ] numba https://github.com/numba/numba/issues/8304
  • [ ] pyarrow
  • [ ] scikit-learn
  • [ ] pydantic
  • [x] cloudpickle
  • [x] upgrade cython to 0.29.32
  • [ ] tensorflow https://github.com/tensorflow/tensorflow/issues/58032
  • [ ] torch https://github.com/pytorch/pytorch/issues/86566
  • [ ] miniconda https://github.com/conda/conda/issues/11170
    • [ ] docker/base-deps/Dockerfile
• [x] claim to support 3.11 in setup.py
• [ ] cicd
  • [ ] .buildkite/
    • [ ] .buildkite/pipeline.build.yml
  • [ ] ci/
    • [ ] ci/build/test-wheels.sh
    • [ ] ci/build/build-docker-images.py
  • [ ] release tests
  • [ ] docker/retag-lambda/python_versions.txt
  • [ ] download_wheels.sh
  • [ ] wheels
    • [ ] python/build-wheel-macos.sh
    • [ ] python/build-wheel-windows.sh
• [ ] Tests
  • [ ] pytest ray/serve/tests
  • [ ] python python/ray/serve/examples/echo_full.py
  • [ ] bazel test //:core_worker_test
  • [ ] bazel test --build_tests_only //:all
  • [ ] //python/ray/tests:test_pydantic_serialization https://github.com/tiangolo/fastapi/issues/5048
  • [ ] //python/ray/train:test_torch_utils
• [ ] Documentation
  • [x] installation.rst

Current status: pip install works locally for python3.11 with modifications. Need to make sure it builds across all versions and tests pass.

Related issue number

#27881

Checks

• [x] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• [x] I've run scripts/format.sh to lint the changes in this PR.
• [x] I've included any doc changes needed for https://docs.ray.io/en/master/.
• [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • [ ] Unit tests
  • [ ] Release tests
  • [ ] This PR is not tested :(

[csko]

Some success building locally:

(venv) csko@cskos-MacBook-Air python % pip install -e . --verbose
...
Successfully installed ray-3.0.0.dev0
(venv) csko@cskos-MacBook-Air python % python
Python 3.11.0 (main, Oct 26 2022, 19:06:18) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ray
>>>

[rickyyx]

IIUC, we are not able to build python 3.11 wheels for mac as well, since our mac CI machines currently don't have python3.11 installed. The newest mac AMIs of 10.15 are amzn-ec2-macos-10.15.7-20221005-230420, which I believed don't include python3.11 as well.

cc @krfricke, @simon-mo - Any thoughts on unblocking wheels built for 3.11 on mac?

[rickyyx]

So the wheel grep seems to work on mac! Could you remove the other "commented out" dead code? And I think we should be able to merge it.

[csko]

• updated PR with a description of the major changes
• merged with upstream
• removed commented out code

[rickyyx]

Thanks so much. LGTM!

A few notes on test for other reviewers:

1. Lint check is soft failing since the wheel from master branch hasn't been uploaded, so the URL in the doc is not valid yet. This should be auto-resovled once the PR is merged and CI runs on the master.
2. Previous mac run see here: https://buildkite.com/ray-project/oss-ci-build-pr/builds/8659#01858375-6bb5-4733-b638-4a1d2c5a25ea
3. I have manually tested the built wheels on Linux with Ray quick start installation, and we are tracking some incompatibility here: https://github.com/ray-project/ray/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Python+3.11+%22

[pcmoritz]

I checked that wheels are only uploaded after the PR is merged -- let's try to merge it :)

[pcmoritz]

After merging and the master build finished, the link is now indeed working 🎉

[grizzlybearg]

Are there any plans for windows python 3.11 wheels?