ml-commons icon indicating copy to clipboard operation
ml-commons copied to clipboard

[Enhancement] Support more Action types in ML Extensibility Framework

Open Zhangxunmt opened this issue 2 years ago • 3 comments

Typically Remote model server support multiple APIs like remote inference, healthy check, or training, etc. Currently in Ml-commons we only support the "predict" API which maps to the "remote inference" in the remote model servers. We should add more APIs in Ml-Commons and map the new APIs to invoke remote servers for other actions like "health check", "training", etc.

Zhangxunmt avatar Aug 22 '23 22:08 Zhangxunmt

I'm not familiar with "health check". Can you give an example? Maybe using OpenAI or SageMaker?

austintlee avatar Aug 22 '23 23:08 austintlee

For example, this is health check API in pytorch https://pytorch.org/serve/inference_api.html. Other platforms has similar APIs like model status, model meta, etc.

Zhangxunmt avatar Aug 25 '23 16:08 Zhangxunmt

will have progress when the batch inference project gets in progress

mingshl avatar Mar 12 '24 17:03 mingshl

This is a similar issue with https://github.com/opensearch-project/ml-commons/issues/2488 and https://github.com/opensearch-project/ml-commons/issues/2484. Closing this one as these two new issues have more details.

Zhangxunmt avatar Jun 18 '24 18:06 Zhangxunmt