Suman Tatiraju

icon location Suman Tatiraju

Results 17 comments of Suman Tatiraju

Questions about input and output shape in model configuration when batch size is 1

@tanmayv25 can you help here?

Cant build python+onnx+ternsorrtllm backends r24.04

Tracking ticket: [DLIS-6397]

TensorRT 8.6.3.1 package in Python PyPy for Triton Nvidia Inference Server version > 24.01

@mc-nv can you take a look?

Inference in Triton ensemble model is much slower than single model in Triton

opened [DLIS-6702]

repeated answer:When I use vllm with Qwen-7b-chat the generated text is x lnot end until the maength, with the repeated answer

Sorry it is difficult to understand your question here. Can you post us what the expected result is?

Memory over 100% with decoupled dali video model

[6864]. created for tracking

Add TT-Metalium as a backend

[6865] created for tracking.

CUDA runtime API error raised when using only cpu on Mac M3

[6862] created to investigate

Custom backend using recommended.cc not generating correct output

[DLIS-6788] created

When the request is large, the Triton server has a very high TTFT.

[6863] created to track