Gaetan RACIC

Hello there! Any news on this PR ? I'm also facing OOM because the inputs are too long. I would also like to be able to set the maximum length...

Hi @craigwalton, I'm facing the same issue as you. In the meantime, I've created my own NuGet which is working. But it is not productized. It seems this project is...

Thanks for your answer 👍

Hello @msaroufim ! Thanks for the new CI. I opened a PR few months ago for the same feature request but did not manage to pass the CI. We are...

It is not clear to me why testModelWithCustomPythonDependency fails

> @nateagr Thank you for the PR. From customer perspective, users only care about the overall inference latency. Internally TS can have multiple phases/queue (eg. in workflow). I think your...

> @nateagr I suggest using "inferenceTimeout" to measure the overall end-to-end timeout. For example, user specifies "inferenceTimeout= 30sec", an inference request will be terminated as long as this request exceeds...

Hi @lxning, I'm going to rephrase what you are suggesting just to make sure we are in phase: we introduce a new parameter "inferenceTimeout" that we use to terminate/abort prediction...