ray-project
[Feature] Support all HTTP request in apiserver V2
Search before asking
- [x] I had searched in the issues and found no similar feature requirement.
Description
In the Apr.22, 2025 community meeting, kuberay main contributors and apiserver main customers have reached an agreement to work on apiserver V2, which implement a transparent receive and forward HTTP server. For details, please refer to https://docs.google.com/presentation/d/1mIBSln1Sb4vphXoYPzL5ygA4i9kz78WBwWX3M3DP-mU/edit?pli=1#slide=id.g34d9b3d41af_0_37
To fully support it before v1.4 release, we need to support ALL existing HTTP requests and add corresponding env test. @rueian has setup the framework for http server and envtest: https://github.com/ray-project/kuberay/pull/3494
Use case
No response
Related issues
No response
Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
Hi @dentiny, I’m interested in the apiserver v2 work. Would it be possible for me to get involved or help with any parts of this effort?
Hi @dentiny, I’m interested in the apiserver v2 work. Would it be possible for me to get involved or help with any parts of this effort?
Thank you @nadongjun for the interest! I will split into subissue after RueiAn's PR gets merged.
The first subissue: #3521
Hi @rueian , if you don't mind, could you please fire the issue as a github subissue? It's a feature supported by github.
The benefit of which is,
- These issues are tracked automatically in the list form;
- The big issue displays the completion progress in X/Y, which is more straightforward.
Hi @dentiny, I'd love to but I don't see that button in KubeRay. I guess the feature is only available for people with a higher permission within the repository.
Hi @dentiny, I'd love to but I don't see that button in KubeRay. I guess the feature is only available for people with a higher permission within the repository.
You should be able now, let's track via sub-issues :stuck_out_tongue_winking_eye:
@rueian @dentiny do you have any issue for @nadongjun to work on? Thanks!
@rueian @dentiny do you have any issue for @nadongjun to work on? Thanks!
Functionality wise, this should be the last piece for HTTP request handling and start HTTP server: https://github.com/ray-project/kuberay/issues/3596
Another thing might be (again), timeout and retry, which is similar to this V1 issue: https://github.com/ray-project/kuberay/issues/3344
For V1, the timeout / retry mechanism is applied at grpc layer, because all HTTP requests are routed to grpc service via reverse proxy (https://grpc.io/blog/coreos/#a-grpc-application-called-echoservice); but V2 doesn't play with grpc any more.
@rueian @dentiny do you have any issue for @nadongjun to work on? Thanks!
Non-coding-wise, which I think even more important to the project, is to prepare a migration doc, though we define it as part of the phase-2 task here.
@dentiny Thanks for the context! I’m happy to help — feel free to assign it to me if needed, and I can start working on it.
Hi @nadongjun thanks for the interest and passion!
I created these two issues:
- timeout / retry: https://github.com/ray-project/kuberay/issues/3606
- migration doc: https://github.com/ray-project/kuberay/issues/3607
Feel free to comment if you're interested, welcome to discuss any time!
@nadongjun Just FYI, another work item is template features with HTTP middleware functions, which is also stage-2 features. I would prefer to leave it after v1 release and verify no issue, because it's a feature / functionality change, documentation / reliability is more important as of now IMO.