multi-model-server
multi-model-server copied to clipboard
awslabs
Multi Model Server is a tool for serving neural net models for inference
https://github.com/awslabs/mxnet-model-server/blob/master/plugins/endpoints/src/main/java/software/amazon/ai/mms/plugins/endpoint/ExecutionParameters.java#L26 'max_request_size' seems to refer to bytes, not mb.
My input image is [batch, 3, None, None], how to represent any size in the signature.json file. I think a solution is to remove image.resize(img_arr, w, h) in mxnet_vision_servion_service.py. But...
I'm getting bad inference results from ResNet ONNX models. The image of the cat used in the example is reported as a pool table when using ResNet v1 or v2....
Currently MMS_METRICS that are logged include things like CPUUtilization and Memory usage. It would be great if these also included things like GPU usage and memory. This would make it...
Having worked in a number of AI startups the biggest limitation of the server is that it can only manage modes on GPUS which are on a single server implementation....
Follow up to https://github.com/awslabs/mxnet-model-server/issues/732#issuecomment-470214725 and https://github.com/awslabs/mxnet-model-server/issues/732#issuecomment-470224017 #### Short Background Currently, there are 2 ways to register models: - POSTing the model to the management API - Writing the model to...
Check if there are better JVM config values that can be used. There are guidelines published by oracle for JVM GC configurations in production setup https://docs.oracle.com/cd/E40972_01/doc.70/e40973/cnf_jvmgc.htm#autoId0
It would be helpful if model-archiver could exclude directories or files when creating an archive. Consider the use case given this file structure: `models/ test/ mms_server.py build/ ` If I...
Jetsons do not use nvidia-smi, hence inaccurate gpu count, 0 instead of one.
It would be great to have a swagger client UI to manage the number of workers etc. It is fairly easy to integrate. See: https://swagger.io/tools/swagger-ui/