coddderX

感觉效果差的好离谱

This may be because the validation dataset is loaded in memory on every executor. Therefore, a large validation dataset may cause out of memory errors

From the source code perspective, validateData is broadcasted to each executor without any compression processing, which leads to a significant consumption of memory. Therefore, currently, the only way to avoid...


 This may be because the LightGBM C++ library does not support streaming validation datasets, but does support streaming training datasets. Therefore, I think the room for improvement in SynapseML...