ray
ray copied to clipboard
[Serve] Add ray serve request timeout to config
Why are these changes needed?
Currently the only way to config the http request timeout is through setting up RAY_SERVE_REQUEST_PROCESSING_TIMEOUT_S (or yet to deprecated SERVE_REQUEST_PROCESSING_TIMEOUT_S) environment variables. This PR adds new config request_processing_timeout to the http_options in serve config and use it as the timeout in the http requests.
- added
RAY_SERVE_REQUEST_PROCESSING_TIMEOUT_Sdeprecation message - added
request_processing_timeouttoHTTPOptionsSchemafor documentation should also generated at https://docs.ray.io/en/master/serve/api/doc/ray.serve.schema.HTTPOptionsSchema.html#ray.serve.schema.HTTPOptionsSchema.request_processing_timeout when merged - Fixed a small bug in when running
serve runwith config files, thehttp_optionsare not picked up - Also did a manual test locally running
serve run test_config.yamland seeing therequest_processing_timeoutin the config getting respected
Related issue number
Closes #33980
Checks
- [ ] I've signed off every commit(by using the -s flag, i.e.,
git commit -s) in this PR. - [ ] I've run
scripts/format.shto lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I added a
method in Tune, I've added it in
doc/source/tune/api/under the corresponding.rstfile.
- [ ] I've added any new APIs to the API Reference. For example, if I added a
method in Tune, I've added it in
- [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
- [ ] Unit tests
- [ ] Release tests
- [ ] This PR is not tested :(