Support for multiple endpoints (routes)

[marcindulak]

Does the cog framework support multiple endpoints (routes) served by a single container?

I see some use cases for serving on multiple endpoints:

1. Container deployment platforms make use of health probe endpoints: for example, Kubernetes defines /livez and /readyz endpoints, Azure Container Apps is based on the Kubernetes approach, AWS ECS is based on Dockerfile HEALTHCHECK, Nomad supports consecutive checks. While the behavior of health probes needs to be considered in the context of the full system, container images which support custom health endpoints are more portable. Other cases for multiple endpoints include monitoring, like the /metrics endpoint of prometheus.

2. Serving multiple models from a single container. A common case would be to serve both /predict and /explain endpoints from the same container, like in pytorch serving, but a need to other endpoints may also appear.

I don't see a support for custom endpoints in cog docs. For comparison custom endpoints are supported by bentoml.

A slightly related issue is discussed in https://github.com/replicate/cog/issues/604, but it concerns the name of the /predictions endpoint and not multiple endpoints.