easytorch icon indicating copy to clipboard operation
easytorch copied to clipboard

Slow recursive file split.

Open aashis-ns opened this issue 2 years ago • 0 comments

split_ratio = np.array(split_ratio) ix = np.argsort(split_ratio) ixx = np.argsort(ix) sm = (np.cumsum(split_ratio[ix])[:-1] * len(files)).astype(int) split = np.split(files, sm) for key, data_ix in zip(['train', 'validation', 'test'], ixx): data[key] = split[data_ix].tolist()

aashis-ns avatar Jun 14 '22 01:06 aashis-ns