bug: bentoml.ray.deployment is broken with Ray 2.8.0

[jerryharrow]

Describe the bug

The implementation of src/bentoml/_internal/ray/init.py had dependencies upon internal Ray object which may change across releases leading to incompatibilities. In particular it references ASGIHTTPSender (which was only present before ray 2.5), https://github.com/bentoml/BentoML/pull/4191 enabled support for Ray 2.5. This has now broken again for Ray 2.8 because there is neither a BufferedASGISender class which is referenced in src/bentoml/_internal/ray/init.py and grpc-client/bentoml/_internal/ray/init.py in the following code fragment.

if pkg.pkg_version_info("ray")[:2] >= (2, 5):
    from ray.serve._private.http_util import BufferedASGISender as ASGIHTTPSender
else:
    from ray.serve._private.http_util import ASGIHTTPSender

_get_service_deployment (http & grpc) should be implemented in a manner that doesn't use Ray private interfaces to make this resilient to Ray version changes. Hopefully, that isn't impossible.

To reproduce

Follow instructions at https://docs.bentoml.org/en/latest/integrations/ray.html with Ray 2.8.0 installed

Example stack with Ray 2.8.0

   module = importlib.import_module(module_name)
  File "/usr/local/Cellar/[email protected]/3.9.18_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/Users/rcorujo/bento_testing/./bento_ray.py", line 8, in <module>
    classifier = bentoml.ray.deployment('mnist_classifier:oxk7lydkxkq2jlg6')
  File "/Users/rcorujo/.virtualenvs/aioli/lib/python3.9/site-packages/bentoml/_internal/utils/lazy_loader.py", line 69, in __getattr__
    self._module = self._load()
  File "/Users/rcorujo/.virtualenvs/aioli/lib/python3.9/site-packages/bentoml/_internal/utils/lazy_loader.py", line 48, in _load
    module = importlib.import_module(self.__name__)
  File "/usr/local/Cellar/[email protected]/3.9.18_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/Users/rcorujo/.virtualenvs/aioli/lib/python3.9/site-packages/bentoml/ray.py", line 1, in <module>
    from ._internal.ray import deployment
  File "/Users/rcorujo/.virtualenvs/aioli/lib/python3.9/site-packages/bentoml/_internal/ray/__init__.py", line 11, in <module>
    from ray.serve._private.http_util import BufferedASGISender as ASGIHTTPSender
ImportError: cannot import name 'BufferedASGISender' from 'ray.serve._private.http_util' 

Expected behavior

Successful startup

Environment

bentoml: 1.1.7 python: 3.9 ray: 2.8.0