Nikhil Kulkarni
Nikhil Kulkarni
*Issue #, if available:* 1. The current notebook breaks due to a bug in the inference toolkit, that processes only one responses instead of all, while that is being fixed,...
There are 3 places still using LOG_LOCATION as system properties: ConfigManager.dumpConfigurations TestUtils launcher.gradle log4j2 does not work with Java system properties, and instead relies on environment variables. These references can...
*Issue #, if available:* This PR fixes the issue described here in PT inference toolkit repo, since the fix can be applied at the transform function in the sagemaker inference...
Basing this issue on a similar issue for TorchServe, reported from a customer: https://github.com/pytorch/serve/issues/1862 Request: MMS needs to verify the [pid](https://github.com/awslabs/multi-model-server/blob/master/mms/model_server.py#L45) is a real MMS process. Motivation: This feature is...
*Issue #, if available:* *Description of changes:* 1. Starting Triton version 23.12, sagemaker tritonserver is available in a different set of accounts, similar to other model servers such as djl-serving,...
Before or while filing an issue please feel free to join our [ slack channel](https://join.slack.com/t/mms-awslabs/shared_invite/enQtNDk4MTgzNDc5NzE4LTBkYTAwMjBjMTVmZTdkODRmYTZkNjdjZGYxZDI0ODhiZDdlM2Y0ZGJiZTczMGY3Njc4MmM3OTQ0OWI2ZDMyNGQ) to get in touch with development team, ask questions, find out what's cooking and more!...
Running the following command currently gives the error: `python launcher.py --enable_tcmalloc --enable_iomp --ninstances=2 -- src/main.py --info config model=bert-base-cased batch_size=16 sequence_length=512` ``` Traceback (most recent call last): File "/home/ubuntu/tune/launcher.py", line 35,...
## Description ## 1. Update TRT Container to build with Triton 24.04. One of the motivations for this is to check if it addresses a segfault issue.