serve
serve copied to clipboard
pytorch
Serve, optimize and scale PyTorch models in production
Although, according to a statement towards the beginning of the tutorial, paths were supposed to relative to the repository's root's parent, most were actually relative to the repository's root. The...
We've been assuming so far that Torchserve can only work with Pytorch Eager mode or Torchscripted models but our current handler is general enough to make it possible to support...
### 🚀 The feature It would be great if, given a handler, we could define multiple serving signatures and bind them to specific endpoints of models. That way, we could...
I'v been looking in previous issues, but I could not find satisfying answer. I have packed model using model-archiver in docker. ```bash docker run --rm -it --name mar -v $(pwd)/output:/output...
@lxning @sharvil # Request level error codes and batch responses Right now the way we handle batch responses is all clients get the same inference for a batch. This is...
## Context * torchserve version: 0.3.0 * torch-model-archiver version: 0.3.0 * torch version: 1.7.1+cpu * torchvision version [if any]: 0.8.2+cpu * Operating System and version: Ubuntu 20.04 ## Your Environment...
### 🚀 The feature I've had this conversation offline a few times so I figured I would summarize it a single thread to explain how batching works in TorchServe today...
### 📚 The doc issue `torchserve` apparently requires Java (and, in particular, Java 17) in order to run. However, neither the [README](https://github.com/pytorch/serve/blob/master/README.md#install-torchserve) nor the [Getting Started doc](https://github.com/pytorch/serve/blob/master/docs/getting_started.md) describe this requirement....
## Description Please include a summary of the feature or issue being fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. Fixes...
## Description Please include a summary of the feature or issue being fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. Fixes...
