fastertransformer_backend
fastertransformer_backend copied to clipboard
what is the use of preprocessing & postprossing ? can i start fastertransformer only for bloom model ?
Description
i fail to start preprossing and postprossing. so i start fastertransformer only, it works fine but the model performance is bad. so i wonder if the reason is the missing of preprossing and postprossing ?? what is the use of preprocessing & postprossing ??
Reproduced Steps
/tritonserver/bin/tritonserver --model-repository=triton-model-store/bloom/
preprocessing: string -> encoded input_ids bloom (fastertransformer): encoded input_ids -> encoded output_ids postprocessing: encoded output_ids -> string
This could be fine if there is no need to run an end-to-end case.