fastertransformer_backend icon indicating copy to clipboard operation
fastertransformer_backend copied to clipboard

Published 20 hours ago • verified publisher icon triton-inference-server

is_return_log_probs is required for decoupled model?

Open flexwang opened this issue 1 year ago • 0 comments

Looks like if is_return_log_probs is set to False, then the decoupled model does not return anything.

flexwang avatar Aug 09 '23 00:08 flexwang