RAM consumption of TimeSeriesDataset

[nicocheh]

I have a dataframe that consumes aprox 10 G in memory. when i try to build the TimeSeriesDataset, it consumes >30G in memory (making explode my RAM). I know It makes sense because the time series dataset is a bigger structure than the dataframe.

How much can the memory consumption grow when building the time series dataset? Like a 4x? I would like to have an estimation to know how to reduce the original dataframe. Is there any way to make TimeSeriesDataset consume less RAM?

Thanks @jdb78

[nicocheh]

@jdb78 Using smaller datasets, i realized that the RAM consumption gets really high and after that it goes down. Do you have any clue of which operation of TimeSeriesDataset can cause this? Do you think it could be optimized to use less RAM?

[Fryman420]

I have the same problem

[binhna]

I am facing the same problem, not sure why, even after decreasing the max_encoder_length. I noticed that after the first epoch was finished, the memory usage increased dramatically.