serve

serve copied to clipboard

Introduce queue timeout for prediction requests

9

## Description This PR follows this feature request https://github.com/pytorch/serve/issues/1322 The feature request is not related to a bug. But at my company, we have this following problem: we would like...

nateagr

Modularize `base_handler.py` into `handler.utils`

1

## Modularize `base_handler.py` into `handler.utils` This is a follow-up note on #1440 where we initially noted a few problems that were making it difficult to add code to TorchServe Initially,...

msaroufim

Health check for all API endpoints

1

### 🚀 The feature At present, there is a health check /ping only for the inference endpoint. Add health check for other endpoints. ### Motivation, pitch There is currently no...

heatxg

NVML_ERROR_NOT_SUPPORTED exception

6

### 🐛 Describe the bug Sometimes it can occur that NVML does not support monitoring queries to specific devices. Currently this leads to failing the startup phase. ### Error logs...

lromor

Model Management API doesn't work with S3 presigned URL

7

## Context I am trying to use the management API with a S3 presigned URL to download and register a new model. This is the code snippet: ``` def create_presigned_url(self,...

Iron-Stark

### 🚀 The feature We're close to fully automated releases with our new nightly build setup by leveraging nightly builds. The remaining gaps are * [x] #1620 * [ ]...

msaroufim

feat(model-archiver): Enabled the built-in base handler

2

## Description This handler is helpful for structural data and raw number arrays. When serving the model, the base handler should be used with an appropriate service envelope ("body", "json"...

Ark-kun

Serving Huggingface Transformers using TorchServe official demo can't run successfully

8

### 🐛 Describe the bug When I ran the official demo Serving Huggingface Transformers using TorchServe - Sequence Classification, I got the error logs as follows. ### Error logs ```...

choshiho

Confused about Cumulative Inference Duration vs. PredictionTime

3

### 📚 The doc issue I am running a model on TorchServe and I am trying to see how long it takes for inference. If I use logging and view...

Hegelim

curl 404 ResourceNotFoundException

3

Hello, I am stuck with an error that I am not sure what does it mean. when I do `curl "http://localhost:8080/models"` I get : `{ "code": 404, "type": "ResourceNotFoundException", "message":...

ma-batita