fastertransformer_backend icon indicating copy to clipboard operation
fastertransformer_backend copied to clipboard

what is the use of preprocessing & postprossing ? can i start fastertransformer only for bloom model ?

Open flyingjohn opened this issue 1 year ago • 1 comments

Description

i fail to start preprossing and postprossing. so i start fastertransformer only, it works fine but the model performance is bad. so i wonder if the reason is the missing of preprossing and postprossing ?? what is the use of preprocessing & postprossing ??

Reproduced Steps

/tritonserver/bin/tritonserver --model-repository=triton-model-store/bloom/

flyingjohn avatar Sep 22 '23 08:09 flyingjohn

preprocessing: string -> encoded input_ids bloom (fastertransformer): encoded input_ids -> encoded output_ids postprocessing: encoded output_ids -> string

This could be fine if there is no need to run an end-to-end case.

yeahdongcn avatar Sep 22 '23 09:09 yeahdongcn