Expose OpenAPI docs

[adriangonz]

trafficstars

MLServer uses FastAPI under the hood, which has some built-in functionality to generate and expose OpenAPI documentation on how to interact with the API. Additionally, the OpenAPI for the standard V2 data plane is also present in the ./openapi/dataplane.yaml file.

It would be good to leverage both points above, to expose the docs directly from an endpoint.

[martapodarta]

Hi Adrian, just to let you know I am working on this issue and I will submit a fix for it soon.

[adriangonz]

Hey @martapodarta , that's amazing!

Thanks for the heads up and for taking the time to look into this one! :rocket:

I haven't dived too much into the available options, but I think FastAPI already had a way to expose a set of OpenAPI docs? It would be great to hear your thoughts on how you're planning to approach this one!

[martapodarta]

Hi Adrian, my idea is to use fastapi.openapi.utils.get_openapi utility function to expose OpenAPI documentation (https://fastapi.tiangolo.com/advanced/extending-openapi/) and as you suggested in the ticket leverage ./openapi/dataplane.yaml file (as well as /openapi/model_repository.yaml) to get the information about endpoints for the OpenAPI schema.

[martapodarta]

Hi Adrian,

I am working on this issue and I was hoping to get your input. In order to extract information from OpenAPI yaml files located in the project (under ./openapi) I need to match API paths and schema objects defined in these files with paths and schema objects defined in OpenAPI schema generated by FastAPI from MLServer code. The problem is that due to naming conventions used in yaml files they don’t match exactly. I would need to normalize these elements which makes code more complex. These are examples that picture the issue:

1. Path

• Dataplane.yaml: ‘/v2/models/${MODEL_NAME}/versions/${MODEL_VERSION}/infer'
• MLServer openapi schema: ‘/v2/models/{model_name}/versions/{model_version}/infer’

2. Path - trailing ‘/’

• Dataplane.yaml: ‘/v2/’
• MLServer openapi schema: ‘/v2’

3. Path/Parameters/name:

• Dataplane.yaml: ‘MODEL_NAME’, ‘MODEL_VERSION’
• MLServer openapi schema: ‘model_name’, ‘model_version’

4. Components/Schemas/Schema_Object:

• Dataplane.yaml: ‘inference_request’
• MLServer openapi schema: ‘InferenceRequest’

5. Schema #ref field:

• Dataplane.yaml: ‘#/components/schemas/inference_request’
• MLServer openapi schema: ‘#/components/schemas/InferenceRequest’

Instead of handling these differences in the code could I modify yaml files itself to match MLServer OpenAPI schema naming and avoid extra complexity in the code? I was also wondering about a dollar sign in API path if it’s intentional, as it doesn’t usually appear in open API spec YAML files and is not mentioned in the OpenAPI spec itself.

[adriangonz]

Hey @martapodarta ,

Those are great points!

Initially, those conventions were taken from the original spec of the V2 Inference Protocol. However, I agree that some of those are not very usual for OpenAPI.

Given that OpenAPI is mainly informative (e.g. as opposed to the protobufs, which need to match exactly), I think it would be great if you could update them to match common conventions (e.g. like not using the dollar signs for variables, replacing capital for lower-cased parameter names, etc.)!

On top of easing the implementation of this particular one, it would also probably help with readability of the spec. :rocket:

[martapodarta]

Hi Adrian,

Thank you for your input. I updated those elements in the yaml files both dataplane.yaml and model_repository.yaml. I also would like to mention few other points and how I am planning to address them.

1. Descriptions of some API paths in dataplane.yaml does not exactly describe MLServer API paths even though the paths itself would match. For example:

• From dataplane.yaml paths/ /v2/models/${MODEL_NAME}/versions/${MODEL_VERSION}/ready Normalized to: '/v2/models/{model_name}/versions/{model_version}/ready' Description for that path is following: "The “model ready” health API indicates if a specific model is ready for inferencing. The model name and (optionally) version must be available in the URL. If a version is not provided the server may choose a version based on its own policies.""

• In MLServer code this would match following FastAPI endpoint: '/v2/models/{model_name}/versions/{model_version}/ready' Model version is a path parameter and is not optional but required.

2. Some paths which are defined MLServer code do not exist in dataplane.yaml For example:

• /v2/models/{model_name}/ready
• /v2/models/{model_name}/infer
• /v2/models/{model_name}

3. OpenAPI schema defined in model_repository.yaml even though it is matching MLServer API paths it does not contain descriptions of those paths.

My intention is to provide a mechanism that will allow for an extraction of existing descriptions from both dataplane.yaml and model_repository.yaml and further exposing this information in MLServer OpenAPI schema but in the situations where the descriptions do not exactly match, they are unavailable or paths are not defined, I would assume that they would be corrected or added to the yaml files itself and maintained in that files. Let me know if you have any comments or questions about this?

[adriangonz]

Hey @martapodarta ,

Wow, that sounds like really great progress!

You raise a very good point. I think the core issue for 1. and 2. is mainly that model_version is optional. However, on the OpenAPI spec just mentions that on the description (and the FastAPI paths just list those as separate endpoints). Would there be a better way to capture this on both the spec and the FastAPI paths?

On the last point, happy to add the description to the model repository API spec. It'd be mainly taken from the relevant docs on the Triton repo. Would that mess up with any WIP you may already have (i.e. in terms of conflicts, etc.)?

[martapodarta]

Hi Adrian,

I should be fine with merging descriptions into model_repository.yaml file once you add them. Thanks again for your input

[martimors]

~~Hi guys! This feature would be absolutely amazing.~~

We already have it, do'h!