[Question] How to run HF model with 1m-length tokens in your exp?

[1649759610]

Hi, @chooper1

We need to use some calibration datasets to do quantization in the exp, but this type of sequence is too long to run in even 80G GPU for HF model.

So I wanna know how do you solve this problem?

Look forward your reply, Thanks!