GUO-QING JIANG issues

Results 6 issues of


                                            GUO-QING JIANG

python_csv.py

Maybe using pandas as csv tutorial will be better?

Can not reproduce the results on the paper with 65B ckpt?

**When we tried to perfrom the Qs in the appendix of the llama paper, we found that it was just repeating... Anything needs to adjust? top_p? temperature?** **Q1: The sun...

[PYTORCH::FP8] FP8 significantly slow down when scaling up to 1000+ GPUs

Small LLMs trained using FP8 with 32 GPUs can achieve 20~30% speed up comparing with bf16. However, scaling up to 1000+ GPUs only achieve less than 5% speed up (TP2...

Token-wise the same generalization?

Is Medusa1 model generalize token-wise the same as the base model w.o. medusa head? I found change medusa choices will change the output.

[Dynamic Batching] Concerns about whether features are not supported using Medusa

I checked the TRT-LLM but found something confusing. There are some features not supported: 1. inferece batch size == 1, (seemed solved recently) 2. not surport in-flight batching, which will...

[Bug] H800 run UT failed.

This setup can not pass UT. Could you please check it ?