serve
serve copied to clipboard
pytorch
Serve, optimize and scale PyTorch models in production
Please have a look at [FAQ's](../../docs/FAQs.md) and [Troubleshooting guide](../../docs/Troubleshooting.md), your query may be already addressed. Your issue may already be reported! Please search on the [issue tracker](https://github.com/pytorch/serve/issues) before creating one....
Hi! I have Wireguard in my machine and have a few other devices connected with it. Let's say my Wireguard IP is 10.0.0.1, then in the `config.properties` file, I change...
Add an example to demonstrate video inferencing where a video payload is sent and predictions are done for each frame. Eg for activity recognition across a video stream
I have two models: model0,model1 and two gpus: gpu0,gpu1. I want to set model0 to gpu0,model0 to gpu1,it means that the work of model0 will always on gpu0 and model1...
Symptom: ab test sent out 11000 requests. The last inference request didn’t get response even though backend processed the request successfully. root cause: [applyAsync](https://github.com/pytorch/serve/blob/master/frontend/server/src/main/java/org/pytorch/serve/workflow/WorkflowManager.java#L381) causes the order of thread completion...
## Context Hi! I have trained maskrcnn with detectron2 and have exported with [this](https://github.com/facebookresearch/detectron2/tree/main/tools/deploy) instructions (I used scripting) torch script model (model.ts). Then I used [this](https://github.com/pytorch/serve/tree/master/examples/object_detector/maskrcnn) example to start a...
## Context I'm trying to increase the configuration parameter `default_response_timeout` to a value higher than 120 seconds but it seems neither the `default_response_timeout` parameter nor the environment variable `TS_DEFAULT_RESPONSE_TIMEOUT` are...
## Is your feature request related to a problem? Please describe. Currently, TorchServe adds headers that prevent from caching the inference results: https://github.com/pytorch/serve/blob/30f83500b0850e26ec55581f48a9307b1986f9f9/frontend/server/src/main/java/org/pytorch/serve/util/NettyUtils.java#L187-L190 This prevents some reverse-proxies like `nginx` from...
Please have a look at [FAQ's](../../docs/FAQs.md) and [Troubleshooting guide](../../docs/Troubleshooting.md), your query may be already addressed. Your issue may already be reported! Please search on the [issue tracker](https://github.com/pytorch/serve/issues) before creating one....
Please have a look at [FAQ's](../../docs/FAQs.md) and [Troubleshooting guide](../../docs/Troubleshooting.md), your query may be already addressed. Your issue may already be reported! Please search on the [issue tracker](https://github.com/pytorch/serve/issues) before creating one....
