Enerai

Generating the .bin file requires much more memory than the expectation

15

For example, a 50G dataset might correspond with a 80G bin file. If I have 128G memory, actually I cannot train the dataset, because during generating the .bin file, the...