deeplearning4j-examples icon indicating copy to clipboard operation
deeplearning4j-examples copied to clipboard

Learning using time series data with infinite stream - continuous learning and normalization

Open kgonia opened this issue 7 years ago • 4 comments

I have question about learning using time series data with infinite stream.

If I had set of historical data and then I want prediction for one step at the time and still improve my network should code look like this:

myModel.fit(historicalData)

// assuming newFeatures is infinite stream, for just represents futher steps
for(INDArray feature: newFeatures){
INDArray timeSeriesOutput = myNetwork.output(feature);

if(iCanGetLabelsForNewData){
// if I want predict labels for n step must wait n steps until I have label for new data
myModel.output(featureWithLabel - n, true);
}
}

Is my aproach correct?

Second problem: how to normalize data with infinite stream when values of features can be much higher than in historical data?

kgonia avatar Dec 22 '17 10:12 kgonia

Have you tried our gitter channel for general questions like this one?

https://gitter.im/deeplearning4j/deeplearning4j

tomthetrainer avatar Jan 02 '18 03:01 tomthetrainer

@KrzychuJedi - did you find out how to do this? I am also interested in achieving this. Thanks

magick93 avatar May 03 '18 04:05 magick93

@magick93 No, I didn't solve this problem.

kgonia avatar May 04 '18 09:05 kgonia

PUt constant that is to be al w a y s applied to any data pull When new data t i c k comes update the constant

januszopechowski avatar May 07 '18 16:05 januszopechowski