chainermn
chainermn copied to clipboard
Add `force_equal_length` flag to `scatter_dataset` method
Current scatter_dataset
creates sub datasets of strictly equal lengths by duplicating some examples when necessary. This is for epoch triggers to work correctly.
However, it is generally unnecessary for validator datasets, and sometimes we want to switch off this behavior, i.e., sometimes we don't want to duplicate some examples.
Therefore, I propose to add force_equal_length
flag to scatter_dataset
, with the default value of True
(to keep the current default behavior).