textrecipes
textrecipes copied to clipboard
Step idea: Sliding window upsample
Suppose you have some long sequences. Have a step that will turn a observation with 1000 tokens into into 10 observations with 100, or more if the user allows overlapping