multi-model-server

multi-model-server copied to clipboard

Hi, I'm trying to load some models at sagemaker endpoint server startup to make them already available on model prediction requests to skip the loading step phase on first request....

sassarini-marco

How to achieve autoscaling when running MMS on a fargate?

1

Hi, I would like to autoscale my model workers based on the request they receive. I am unable to locate any documentation on that. Could somebody please help me configure...

sunilkumarmohanty

Issue: Memory Leak when serving multiple models

4

## Description I am encountering a memory leak when serving multiple MXNet models behind the same endpoint in multi-model-server. I am using 2 docker containers, with the multi-model-server docker image,...

pratikluitel

I was using Intel-based machine and it was running all fine, but now I switched to Apple M1 based Mac Pro, and now the below line giving an error which...

jaiswalvineet

when I use mms , the url is like http:ip:port/my_name/predict. The question is how to change the "predict" to my own words. if I want url is http:ip:port/my_name/predictFile, how can...

xuweidongkobe

command not found: multi-model-server

1

Hi, I am trying with [DUC-ResNet101 (from ONNX model zoo)](https://github.com/awslabs/multi-model-server/blob/master/docs/model_zoo.md#duc-resnet101_onnx) and when I want to start server with the following line of code: `multi-model-server --models duc=https://s3.amazonaws.com/model-server/model_archive_1.0/onnx-duc.mar ` I get this...

nimafo

Process run on a single CPU core

2

I serve the model on CPU and I notice the process only run on a single CPU core (100%), how to config process run on multi core CPU?

ngoanpv

got the following error after setting jsonlayout in frontend/server/src/main/resources/log4j2.xml ERROR Unable to invoke factory method in class org.apache.logging.log4j.core.layout.JsonLayout for element JSONLayout: java.lang.IllegalStateException: No factory method found for class org.apache.logging.log4j.core.layout.JsonLayout java.lang.IllegalStateException:...

lxning

Hi there! We're encountering an issue with MMS and deployment of MXNET models. We thought it was related to the way we're packing the model, but after some digging, it...

llorenzo-matterport

[Q] GPU support

3

AWS documentation (https://docs.aws.amazon.com/sagemaker/latest/dg/multi-model-endpoints.html) tells "Multi-model endpoints are not supported on GPU instance types.". Kindly explain if it is not technically possible or not yet implemented.

oonisim