BiaPy icon indicating copy to clipboard operation
BiaPy copied to clipboard

Training dataset sharding

Open danifranco opened this issue 7 months ago • 0 comments

Now the dataset is replicated for each worker that is spawned. We should split the dataset as it is done in "by chunks" inference in order to save memory when training with large datasets

danifranco avatar Mar 30 '25 13:03 danifranco