ray icon indicating copy to clipboard operation
ray copied to clipboard

[Serve][Doc] Add Batching User Guide

Open simon-mo opened this issue 3 years ago • 1 comments

Why are these changes needed?

Add a new page discussing how to use the batching decorator.

Related issue number

Closes #27445

Checks

  • [x] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • [x] I've run scripts/format.sh to lint the changes in this PR.
  • [x] I've included any doc changes needed for https://docs.ray.io/en/master/.
  • [x] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • [x] Unit tests
    • [ ] Release tests
    • [ ] This PR is not tested :(

simon-mo avatar Aug 10 '22 05:08 simon-mo

Will do a pass after @dmatrix's comments are addressed

edoakes avatar Aug 10 '22 15:08 edoakes

Comments addressed! I think we can keep the tutorial example which uses PyTorch and show case how to compile batches into tensors and others.

simon-mo avatar Aug 11 '22 03:08 simon-mo

@simon-mo you need codeowner approval from someone here

edoakes avatar Aug 11 '22 15:08 edoakes