[WIP] Multiple application support

[sihanwang41]

Signed-off-by: Sihan Wang [email protected]

Why are these changes needed?

Serve API Change

• Add route_prefix and app_name arguments into 'serve.run' API.
• Add serve.delete API.

Serve Agent API Change

• Change deployments API to application API.
• Add delete application API
• Instead of getting one application status, Add get all applications status API. (list of StatusOverview)
• Support multi application deploy API.

Schema Change

• Add app_name in StatusOverview proto.
• Add app_name and route_prefix in ServeApplicationSchema. (per application schema)
• Introduce and promote ServeDeploySchema including list of ServeApplicationSchema for multi app use case. (this is a breaking change, we can add a flag to support both if needed. Or we can ask the user to run serve build to regenerate the deploy config file)

Status Change

• Add DELETING status to ApplicationStatus.

Config file example

# This file was generated using the `serve build` command on Ray v3.0.0.dev0.

host: 0.0.0.0

port: 8000

applications:

- app_name: my_app
  route_prefix: /my_app_path
  import_path: test_schema:g
  runtime_env: {}
  host: 0.0.0.0
  port: 8000
  deployments:
  - name: func_1
  - name: func_2
  - name: func_3
  - name: combine
  - name: DAGDriver
    route_prefix: /
- app_name: my_app2
  route_prefix: /my_app_path2
  import_path: test_schema2:a
  runtime_env: {}
  host: 0.0.0.0
  port: 8000
  deployments:
  - name: A

note:

• host and port attributes per application can be deleted.
• route_prefix per deployment can be deleted.

Related issue number

Checks

• [ ] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• [ ] I've run scripts/format.sh to lint the changes in this PR.
• [ ] I've included any doc changes needed for https://docs.ray.io/en/master/.
• [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • [ ] Unit tests
  • [ ] Release tests
  • [ ] This PR is not tested :(

[edoakes]

host and port attributes per application can be deleted.

Not sure exactly what you mean by this -- it should not be supported at all right?

[edoakes]

@sihanwang41 we will also need a rollout plan for how to make the REST API changes across Serve/Kuberay to avoid breakage

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

• If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[stale[bot]]

Hi again! The issue will be closed because there has been no more activity in the 14 days since the last message.

Please feel free to reopen or open a new issue if you'd still like it to be addressed.

Again, you can always ask for help on our discussion forum or Ray's public slack channel.

Thanks again for opening the issue!