Roman Doronin

Model per instance model-mesh by default

1

Now model load only on one instance, and lazy loading on another pods, when reauest has come. Can we modify internal modelmesh parameters for default loading model on all ServingRuntime...

How to run modelmesh without k8s

1

Can you describe steps please, for running modelmesh locally with runtime adapter, etcd and some serving? It needs for local debugging and clarifying some logic of work

inference with timeouts leads to Internal error

2

ServingRuntime: `torchserve` ### Current behavior * sent requests with client timeouts (load our modelmesh) * after some time, client starts to receive ``` ERROR: Code: Internal Message: org.pytorch.serve.grpc.inference.InferenceAPIsService/Predictions: INTERNAL: Model...