MLDataUtils.jl
MLDataUtils.jl copied to clipboard
CV and kfolds function for time series
Hi,
Is there any function to create indices as in kfolds()
but having them not overlapping and moving from first to last?
I see two ways one would have it:
- K-folds and each fold partitioned: train/valid = 0.9/0.1, for instance,
- K-folds and train size = valid size, thus we need to split the data into K+1 folds
Example 1:
kfolds(1:50; k=5; partition=0.9) # something like this
train: [1:9],[11:19],[21:29],[31:39],[41:49]
valid: [10],[20],[30],[40],[50]
Example 2:
full-sample is 1:120
, and 5-fold result considering equal train/valid sizes would look like
train: [1:20],[21:40],[41:60],[61:80],[81:100]
valid: [21:40],[41:60],[61:80],[81:100],[101:120]
I used 120 because of simpler imagination of splits.
Each example uses its observations for training once. In the first, we even don't use the validation observations to train on them. We might. Also, there can be the third example with expanding partitions, however, we would use first parts multiple time in validation and thus implicitly put more value on them in.
Thanks for comments.