Xiaotian Han

icon indicating a website link https://hanxiaotian.github.io/ icon indicating an email [email protected]

icon location Seattle icon location Research Scientist in Multimodal Large Language Models

Results 1 issues of Xiaotian Han

SFT training doesn't fully go through all samples

3 comment

Current training uses ConstantLengthDataset. This dataset return fixed length of tokens (2048) in every step, however, the total number of steps are calculated based on the number of samples. I...