Naman Nandan

Refactor reqirements handling and model archive creation for TS MME example

21

*Description of changes:* - Refactor Torchserve MME example to use Torchserve `v0.9.0` - Update example to use PyTorch 2.1.0 Sagemaker inference DLC - Fix PyTorch and Cuda version incompatibility in...

Dev docker image with CPP backend support

1

## Description Please read our [CONTRIBUTING.md](https://github.com/pytorch/serve/blob/master/CONTRIBUTING.md) prior to creating your first pull request. Please include a summary of the feature or issue being fixed. Please also include relevant motivation and...

[WIP] Enable generation of AOT compiled artifacts for llama2 on inf2 example

1

## Description Add support for generation of model compiled artifacts ahead of time for the llama2 on inf2 example. ## Type of change - [x] New feature (non-breaking change which...

### 🚀 The feature Periodically dump the updated metrics configuration inclusive of auto detected(#2747) metrics to the metrics configuration file. ### Motivation, pitch As metrics not defined in the metrics...

Upgrade setup-conda from v1 to v1.2.1

1

## Description Upgrade setup-conda from v1 to v1.2.1 Example failures observed with setup-conda v1: https://github.com/pytorch/serve/actions/runs/6404244594/job/17384395423 ``` Getting original pythonLocation /usr/bin/which python Error: The process '/usr/bin/which' failed with exit code 1...

## Description Since configuration options can be set at various levels, i.e command line, TorchServe configuration file, model configuration file etc.., it is useful to include what these configuration values...

## Description Add support for server side gRPC max connection age configuration References: 1. Netty gRPC [maxConnectionAge](https://grpc.github.io/grpc-java/javadoc/io/grpc/netty/NettyServerBuilder.html#maxConnectionAge(long,java.util.concurrent.TimeUnit)) 2. Netty gRPC [maxConnectionAgeGrace](https://grpc.github.io/grpc-java/javadoc/io/grpc/netty/NettyServerBuilder.html#maxConnectionAgeGrace(long,java.util.concurrent.TimeUnit)) 3. Default gRPC [KeepAlive configuration](https://grpc.io/docs/guides/keepalive/#keepalive-configuration-specification) Fixes #2999 ## Type...

### 🚀 The feature Currently, the response for the describe model API does not include all the model configuration options: Describe model API response: https://github.com/pytorch/serve/blob/13d092c002114e5f28d92ac8ad4f21a1a56f2f1a/frontend/server/src/main/java/org/pytorch/serve/http/messages/DescribeModelResponse.java#L10-L25 Model configuration options: https://github.com/pytorch/serve/blob/13d092c002114e5f28d92ac8ad4f21a1a56f2f1a/frontend/server/src/main/java/org/pytorch/serve/wlm/Model.java#L43-L56 Update...

## Description The Mac CI and regression failures are being caused by the following reasons: 1. The `macos-lastest` github runner tag now points to `arm64` based Mac. 2. PyTorch 2.3...

## Description Based on fix demonstrated in https://github.com/pytorch/serve/pull/3267 with additional exception handling. The following cases are handled: 1. If the stream has already been closed by the client, we won't...