Junseo Park

https://company.typecast.ai/en/

Neosapience, Inc. Performance Engineer

Results 2 comments of


                                            Junseo Park

Can tensorrt-llm or how tensorrt-llm support that seprating the prefill stage and decode stage in different GPU or different nodes with self configuration

Is there any update regarding this feature?

does NVIDIA L20 GPUs support FP8 quantization?

> The FP8 FMHA on SM89 (L20) is on going. So, you could only enable FP8 GEMM on L20 now. Do you have an estimate for when the implementation will...