fastertransformer_backend icon indicating copy to clipboard operation
fastertransformer_backend copied to clipboard

Published 20 hours ago • verified publisher icon triton-inference-server

Config.pbtxt for all_models/t5/fastertransformer incorrect

Open dhaval24 opened this issue 2 years ago • 1 comments

Description

The Latest faster transformer v5.1.1 which is being used by the Fastertransformer backend latest release prescribes that T5 decoder output - [output_ids and sequence_length] should be int32 type however in the current config.pbtxt its specified as uint32.

https://github.com/NVIDIA/FasterTransformer/blob/release/v5.1.1_tag/docs/t5_guide.md

Please update the config.pbtxt without correct output configurations.

Reproduced Steps

Run T5 in ensemble mode and the post processing logic will indicate that the output should be INT32.

dhaval24 avatar Nov 14 '22 18:11 dhaval24

We have updated the config.pbtxt of latest v1.4 version.

byshiue avatar Jan 23 '23 03:01 byshiue