flashinfer icon indicating copy to clipboard operation
flashinfer copied to clipboard

[Question] Overflow risks when batch size and sequence length grows extremely large

Open rchardx opened this issue 1 year ago • 1 comments

For example, when batch size is 128, sequence length is like 32K, is there any possibility for FlashInfer's internal computation overflowing?

rchardx avatar Nov 08 '24 10:11 rchardx

I'm not aware of that.

yzh119 avatar Nov 08 '24 23:11 yzh119

Closing based on @yzh119's answer.

sricketts avatar Sep 30 '25 01:09 sricketts