bhsueh_NV
bhsueh_NV
you **should** set `model_checkpoint_path` to `/workspace/build/fastertransformer_backend/all_models/t5/fastertransformer/1/2-gpu/`, **but not** `2_gpu/`
You can change the based triton server docker version in `Docker` file. The default version is `21.08`, which is installed cuda 11.4.
Do you mean that you can build the docker successfully before you modify the version of docker, but encounter the error after you change the docker version? How do you...
Have you tried other version? `21.07`? `21.06`? You can also try to update your driver. Since the error of 20.10 is on github access, I have no idea about it.
Sorry, I have no idea about the error because it does not happen on FT side.
I think the version of triton is too old and is not compatible.
Please set the model-repository=/workspace/build/fastertransformer_backend/all_models/t5
1. The error happens here https://github.com/triton-inference-server/fastertransformer_backend/blob/dev/v1.1_beta/src/libfastertransformer.cc#L552, which means your tensor_para_size_ * pipeline_para_size_ != number of GPU you have. You need to use CUDA_VISIBLE_DEVICES to setup what GPU you want to...
I remove the some installation which leads to error in older version, you can try again.
I think you don't load the model successfully or correctly.