serve
serve copied to clipboard
pytorch
Serve, optimize and scale PyTorch models in production
Hello, I cannot find no documentation on serving the efficientnet(from torchvision) with torchserve. I struggle, as the serve worker dies everything and the status of the server jumps around from...
### 📚 The doc issue There is no documentation about decoding the received bytes form PredictionResponse into torch tensor efficiently. Currently, the only working solution is using `ast.literal_eval`, which is...
## Background Following [the guide](https://github.com/pytorch/serve/blob/master/kubernetes/GKE/README.md##-TorchServe-on-Google-Kubernetes-Engine-(GKE)) to get torchserve set up on GKE and I'm running into an issue installing the helm charts. I got to step three in the [GKE...
## Context I ran the torch model archiver on a different machine to create a mar with a custom handler for transformer model using this command: ``` torch-model-archiver --model-name TranslationClassifier...
Cannot load model with pytorch/torchserve:0.5.3-cpu docker image. The same model is loaded on using pytorch/torchserve:0.5.1-cpu and the docker image built from the master branch. Marfile and config.properties - https://drive.google.com/drive/folders/1vD6HvB0IivS9nvpfRHj4nn6hNyjEukei?usp=sharing ##...
I served waveglow_synthesizer and predict. $ curl http://127.0.0.1:8080/predictions/waveglow_synthesizer -T sample_text.txt -o audio.wav But I get the following error and cannot generate audio. ``` 2022-06-03T07:43:12,809 [INFO ] W-9000-waveglow_synthesizer_1.0-stdout MODEL_LOG - File...
## Is your feature request related to a problem? Please describe. The current [implementation of gRPC](https://github.com/pytorch/serve/pull/687) doesn't support communication over a secure channel. ## Describe the solution Add support for...
## Context * torchserve version: 0.5.3 * torch version: 1.10 * torchvision version [if any]: None * torchtext version [if any]: None * torchaudio version [if any]: None * java...
 Hello guys! I'm trying to serve my face detection model using torchserve docker. I Installed Dockerfile which is working well on RTX 2080, But it is...
### 🚀 The feature Regression test on GPU take about 30 min so could be potentially added to CI which takes about 20 min Regression test on CPU takes more...
