bhsueh_NV
bhsueh_NV
Besides, do you use main branch? I remember we don't need nccl for gpt module.
The dockers we use in the document are open on NGC.
We don't have Dockerfile. We use the docker image of NGC like `nvcr.io/nvidia/pytorch:22.03-py3` directly.
The docker is open in NGC, you could pull it directly.
Close this bug because it is inactivated. Feel free to re-open this issue if you still have any problem.
1. It is tested by random model weight. We launch the triton server and measure the time of queries with different batch size and sequence lengths. More details about setting...
The example is only tested on c directly. It does not contains the overhead of send/recv for serving.
Close this bug because it is inactivated. Feel free to re-open this issue if you still have any problem.
Can you provide more details? How do you install and what error do you encounter? Besides, 22.03 docker image is latest one, we still not verify on it. You can...
Close this bug because it is inactivated. Feel free to re-open this issue if you still have any problem.