LLaDA icon indicating copy to clipboard operation
LLaDA copied to clipboard
verified publisher icon ML-GSAI

Some questions about data processing

Open yuecao0119 opened this issue 9 months ago • 1 comments

Thank you for your excellent work.

I have some questions about data processing and look forward to your response.

  1. Do you use the same data processing scheme as SMDM, that is, packing and padding operations?
  2. And in this operation, are the samples visible to each other? There seems to be no attn mask in the data processing in SMDM.

yuecao0119 avatar Mar 15 '25 09:03 yuecao0119

Thank you for your attention!

  1. Yes.
  2. Yes.

nieshenx avatar May 31 '25 15:05 nieshenx