LLaDA icon indicating copy to clipboard operation
LLaDA copied to clipboard
verified publisher icon ML-GSAI

Does LLADA support batch inference now?

Open yyujinxin opened this issue 6 months ago • 2 comments

Image

Should this “1” be batch_size?

yyujinxin avatar Jun 06 '25 02:06 yyujinxin

Sorry, I sincerely apologize for not providing code related to batch inference at present. If you intend to perform batch inference, it might be necessary for you to debug the design of the attention mask on your own.

nieshenx avatar Jun 08 '25 10:06 nieshenx

Quick follow up on this: why should 1 not be batch_size? If you have suggestions on how to update this, we'd be happy to debug this issue ourselves and suggest a fix.

adebayoj avatar Jun 12 '25 22:06 adebayoj