server

server copied to clipboard

SSD_MobileNetv1_COCO label_filename incorrect classification

9

**Description** When running ssd_mobilenetv1_coco in triton, and specifying the class labels in the model config, it seems that the labels are not assigned correctly. **Triton Information** What version of Triton...

jishminor

Triton server always crash during stress test.

5

**Description** **Triton Information** 22.04 Are you using the Triton container or did you build it yourself? - nvcr.io/nvidia/tritonserver:22.02-py3 - nvcr.io/nvidia/tritonserver:22.04-py3 - nvcr.io/nvidia/tritonserver:22.07-py3 **To Reproduce** Steps to reproduce the behavior. Describe...

kimdwkimdw

Couldn't get temp CUBIN file name - TensorFlow XLA

3

Hi everyone, I am really struggling finding a solution for this problem. It happens when I run the server with TensorFlow Model using the GPUs, but I get this error...

mhbassel

Customize response when using raw binary request.

3

**Is your feature request related to a problem? Please describe.** When using [raw binary request](https://github.com/triton-inference-server/server/blob/main/docs/protocol/extension_binary_data.md#raw-binary-request), the response will include tensors after the metadata response which prevents deserializing the response (the...

N78750469

Migrate OV to 2022.1

1

kthui

[question] Development workflow for custom C++ backends

1

When developing custom backends with triton, the workflow can be fairly slow. Mainly due to having to restart the server each time I want to test something. Usually it can...

jamied157

Python Backend complains "triton_python_backend_utils" has no attribute "InferenceRequest"

12

I'm using the python business logic scripting, and a conda packed python environment with python3.8. Both 22.06 and 22.07 version show the following error message "UNAVAILABLE: Internal: AttributeError: module 'triton_python_backend_utils'...

Michael-Jing

Load Shared Libraries Dynamically

2

Allow user to load TensorRT shared libraries via the backend configuration. Related TensorRT backend ticket: https://github.com/triton-inference-server/tensorrt_backend/pull/25

the-david-oy

Have any roadmap to support http2?

1

**Is your feature request related to a problem? Please describe.** Hope triton server can support http2 protocol.

sunhailin-Leo

health check should not say it's ready when cuda device-side assertion error is triggered

5

**Is your feature request related to a problem? Please describe.** If one of the model triggers a cuda device-side assertion error, all the model instances from that trtis process is...

ghost