multi-model-server
multi-model-server copied to clipboard
awslabs
Multi Model Server is a tool for serving neural net models for inference
There are 3 places still using LOG_LOCATION as system properties: ConfigManager.dumpConfigurations TestUtils launcher.gradle log4j2 does not work with Java system properties, and instead relies on environment variables. These references can...
Hi, MMS by default will print memory utilization into log which is great. The problem I have is after each request to MMS, the memory utilization increment a little bit....
## Issue #, if available: The orphan processes get created when you fire multiple register and unregister calls on same model one after another. The orphan worker processes hogs the...
preload_model=true set in config.properties. send request: POST /models start mms, the following log shows that preload_model is still false when mms sends cmd to backend server.
Hi, Is it possible to write custom plug-in to the MSS? Best regards, Huryan
Hi All, I would change https://github.com/awslabs/multi-model-server/blob/master/plugins/endpoints/src/main/java/software/amazon/ai/mms/plugins/endpoint/ExecutionParameters.java to write to output stream custom JSON with the same url pattern. Is there any possibility to change (override) logic in plugin classes without...
cask option update Before or while filing an issue please feel free to join our [ slack channel](https://join.slack.com/t/mms-awslabs/shared_invite/enQtNDk4MTgzNDc5NzE4LTBkYTAwMjBjMTVmZTdkODRmYTZkNjdjZGYxZDI0ODhiZDdlM2Y0ZGJiZTczMGY3Njc4MmM3OTQ0OWI2ZDMyNGQ) to get in touch with development team, ask questions, find out what's...
Hi AWS team. I love multi-model-server by the way. My team wanted to implement MMS as a backend for Multi-Model Endpoints running on SageMaker, while allowing for custom model handlers...
``` **At the very start I was getting this error** Traceback (most recent call last): 2020-11-10 15:14:25,940 [INFO ] W-get-text-3-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - File "/usr/local/lib/python3.6/dist-packages/mms/service.py", line 108, in predict 2020-11-10 15:14:25,940...
When post a big file(a picture or a video) >=8M, docker will not release memory. when left momory cant support a requst. It will report error:io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16777216...
