deita icon indicating copy to clipboard operation
deita copied to clipboard

Published 20 hours ago • verified publisher icon hkust-nlp

The length of samples

Open Ber666 opened this issue 9 months ago • 1 comments

It seems each sample in the deita dataset consists of a lot of turns and is super long (>10k tokens). Your paper mentioned the max length of input is 2048 for SFT. Does that mean most text of each training sample is truncated and discarded?

Ber666 avatar May 10 '24 05:05 Ber666