openems icon indicating copy to clipboard operation
openems copied to clipboard
Published 1 month ago • verified publisher icon OpenEMS

WIP : Long short term memory prediction models for Time series Data

Open Bishalghimire1997 opened this issue 1 year ago • 2 comments

This Model uses the LSTM (Type of Recurring neural network ) algorithm for predicting the Consumption and Production active power times series data.

Bishalghimire1997 avatar Jul 26 '24 11:07 Bishalghimire1997

Codecov Report

Attention: Patch coverage is 26.48770% with 1643 lines in your changes missing coverage. Please review.

Additional details and impacted files 
@@              Coverage Diff              @@
##             develop    #2721      +/-   ##
=============================================
- Coverage      57.31%   56.60%   -0.71%     
- Complexity      9225     9457     +232     
=============================================
  Files           2188     2241      +53     
  Lines          93247    95482    +2235     
  Branches        6913     7056     +143     
=============================================
+ Hits           53437    54035     +598     
- Misses         37850    39457    +1607     
- Partials        1960     1990      +30

codecov[bot] avatar Jul 26 '24 11:07 codecov[bot]

Very interesting and nice pull request! Do you have benchmarks values?

parapluplu avatar Jul 29 '24 08:07 parapluplu

@Bishalghimire1997 & @pooran-c: Thank you for your amazing work. This is definitely the most complete implementation of LSTM in Java out there. 🚀

I did some final touches here and there in the previous commits to fit with OpenEMS coding guidelines and best practices. Please check them out.

Some parts are still commented out and some others to be discussed and validated in production. I am still merging now to avoid having more merge conflicts in future. We can continue improving in develop from now on and I would also like to invite the (scientific) OpenEMS community to roll some tests. Or even better: I would welcome, if you could introduce the LSTM predictor to the OpenEMS Community on https://community.openems.io/. It was discussed briefly also on the OpenEMS Conference this year.

Some future TODOs here:

  • Is it required to use ArrayLists or would a normal (immutable) list be sufficient? Then we could use simple Stream.toList() everywhere.
  • I usually use ZonedDateTime instead of OffsetDateTime. Not sure which would fit better implications here...
  • Reactivate or remove commented JUnit tests, remove System.out.println()s, etc.

sfeilmeier avatar Dec 05 '24 11:12 sfeilmeier

Hello

I have tried to activate the LSTM predictor but it has been throwing me errors.

Jan 21 10:40:58 raspberrypi java[34549]: =====> Average RMS error for trend = 299.8913298466389 Jan 21 10:40:58 raspberrypi java[34549]: java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException: Index -1 out of bounds for length 0 Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) Jan 21 10:40:58 raspberrypi java[34549]: at io.openems.edge.predictor.lstm.train.TrainAndValidateBatch.<init>(TrainAndValidateBatch.java:69) Jan 21 10:40:58 raspberrypi java[34549]: at io.openems.edge.predictor.lstm.train.LstmTrain.run(LstmTrain.java:102) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.lang.Thread.run(Thread.java:1583) Jan 21 10:40:58 raspberrypi java[34549]: Caused by: java.lang.IndexOutOfBoundsException: Index -1 out of bounds for length 0 Jan 21 10:40:58 raspberrypi java[34549]: at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:100) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:106) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:302) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.Objects.checkIndex(Objects.java:385) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.ArrayList.get(ArrayList.java:427) Jan 21 10:40:58 raspberrypi java[34549]: at io.openems.edge.predictor.lstm.common.HyperParameters.getlastModelSeasonality(HyperParameters.java:800) Jan 21 10:40:58 raspberrypi java[34549]: at io.openems.edge.predictor.lstm.train.MakeModel.trainSeasonality(MakeModel.java:117) Jan 21 10:40:58 raspberrypi java[34549]: at io.openems.edge.predictor.lstm.train.TrainAndValidateBatch.lambda$new$0(TrainAndValidateBatch.java:53) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1760) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) Jan 21 10:40:58 raspberrypi java[34549]: java.io.FileNotFoundException: /home/user/openems/data/lstm/UnmanagedConsumptionActivePower (No such file or directory) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.io.FileOutputStream.open0(Native Method) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.io.FileOutputStream.open(FileOutputStream.java:289) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:230) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:118) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.io.FileWriter.<init>(FileWriter.java:67) Jan 21 10:40:58 raspberrypi java[34549]: at io.openems.edge.predictor.lstm.common.ReadAndSaveModels.save(ReadAndSaveModels.java:59) Jan 21 10:40:58 raspberrypi java[34549]: at io.openems.edge.predictor.lstm.train.TrainAndValidateBatch.<init>(TrainAndValidateBatch.java:76) Jan 21 10:40:58 raspberrypi java[34549]: at io.openems.edge.predictor.lstm.train.LstmTrain.run(LstmTrain.java:102) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.lang.Thread.run(Thread.java:1583) Jan 21 10:40:58 raspberrypi java[34549]: =====> Batch = 2/10 Jan 21 10:40:58 raspberrypi java[34549]: =====> Epoch= 0/10

A few questions do rise: I haven't built the code myself but used the jar file. I have declared the correct location of the data file in the service file (ubuntu). Do you need to use rrd4j in order to feed the algorithm with data? I have all my data stored (from 2 years) in influx.

yvesilknievil avatar Jan 21 '25 09:01 yvesilknievil

@yvesilknievil Yes, you need to have rrd4j for the historical data. Also, check if '...\openems\data\rrd4j\rrd4j0_sum' contains 'UnmanagedConsumptionActivePower'.

We’ll need to verify this before starting operations. We're working on handling this folder and its files to prompt errors early.

pooran-c avatar Jan 21 '25 11:01 pooran-c

Is there a way I can inject my influx data into this? Transform them into rrdj4 compatible files?

yvesilknievil avatar Jan 22 '25 16:01 yvesilknievil

Is there a way I can inject my influx data into this? Transform them into rrdj4 compatible files?

I did not look into the details, but the LSTM predictor uses a generic Timedata reference, so it should transparently work with InfluxDB out of the box.

	@Reference
	private Timedata timedata;

https://github.com/OpenEMS/openems/blob/develop/io.openems.edge.predictor.lstm/src/io/openems/edge/predictor/lstm/PredictorLstmImpl.java#L75-L76

sfeilmeier avatar Jan 23 '25 07:01 sfeilmeier

thanks @sfeilmeier I'll have a look at this. However it seems like the predictor looks for a specific file in the data location. So probably needs some changes.

However, even with rrd4 I get some weird errors that need debugging. I need to have a closer look but any help would be appreciated.

Worker error. RuntimeException: Error in getting prediction execution - Caused by: java.util.NoSuchElementException
Jan 23 07:43:19 nuc01 java[6482]: java.lang.RuntimeException: Error in getting prediction execution
Jan 23 07:43:19 nuc01 java[6482]:         at io.openems.edge.predictor.lstm.PredictorLstmImpl.createNewPrediction(PredictorLstmImpl.java:162)
Jan 23 07:43:19 nuc01 java[6482]:         at io.openems.edge.predictor.api.prediction.AbstractPredictor.getPrediction(AbstractPredictor.java:68)
Jan 23 07:43:19 nuc01 java[6482]:         at io.openems.edge.core.predictormanager.PredictorManagerImpl.getPrediction(PredictorManagerImpl.java:94)
Jan 23 07:43:19 nuc01 java[6482]:         at io.openems.edge.energy.v1.optimizer.UtilsV1.createSimulatorParams(UtilsV1.java:85)
Jan 23 07:43:19 nuc01 java[6482]:         at io.openems.edge.energy.v1.optimizer.OptimizerV1.createParams(OptimizerV1.java:114)
Jan 23 07:43:19 nuc01 java[6482]:         at io.openems.edge.energy.v1.optimizer.OptimizerV1.forever(OptimizerV1.java:62)
Jan 23 07:43:19 nuc01 java[6482]:         at io.openems.common.worker.AbstractWorker$1.run(AbstractWorker.java:155)
Jan 23 07:43:19 nuc01 java[6482]: Caused by: java.util.concurrent.ExecutionException: java.util.NoSuchElementException
Jan 23 07:43:19 nuc01 java[6482]:         at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
Jan 23 07:43:19 nuc01 java[6482]:         at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
Jan 23 07:43:19 nuc01 java[6482]:         at io.openems.edge.predictor.lstm.PredictorLstmImpl.createNewPrediction(PredictorLstmImpl.java:147)
Jan 23 07:43:19 nuc01 java[6482]:         ... 6 more
Jan 23 07:43:19 nuc01 java[6482]: Caused by: java.util.NoSuchElementException
Jan 23 07:43:19 nuc01 java[6482]:         at java.base/java.util.ArrayList$Itr.next(ArrayList.java:1052)
Jan 23 07:43:19 nuc01 java[6482]:         at java.base/java.util.Collections.min(Collections.java:625)
Jan 23 07:43:19 nuc01 java[6482]:         at io.openems.edge.predictor.lstm.common.HyperParameters.getMinimumErrorModelSeasonality(HyperParameters.java:684)
Jan 23 07:43:19 nuc01 java[6482]:         at io.openems.edge.predictor.lstm.common.HyperParameters.getBestModelSeasonality(HyperParameters.java:766)
Jan 23 07:43:19 nuc01 java[6482]:         at io.openems.edge.predictor.lstm.common.LstmPredictor.predictSeasonality(LstmPredictor.java:42)
Jan 23 07:43:19 nuc01 java[6482]:         at io.openems.edge.predictor.lstm.PredictorLstmImpl.predictSeasonality(PredictorLstmImpl.java:281)
Jan 23 07:43:19 nuc01 java[6482]:         at io.openems.edge.predictor.lstm.PredictorLstmImpl.lambda$createNewPrediction$0(PredictorLstmImpl.java:133)
Jan 23 07:43:19 nuc01 java[6482]:         at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
Jan 23 07:43:19 nuc01 java[6482]:         at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1760)
Jan 23 07:43:19 nuc01 java[6482]:         at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
Jan 23 07:43:19 nuc01 java[6482]:         at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
Jan 23 07:43:19 nuc01 java[6482]:         at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
Jan 23 07:43:19 nuc01 java[6482]:         at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
Jan 23 07:43:19 nuc01 java[6482]:         at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)
Jan 23 07:43:19 nuc01 java[6482]: 2025-01-23T07:43:19,802 [1062-176] INFO  [edge.core.user.UserServiceImpl] Authentication successful with password only for user [Admin].
Jan 23 07:43:19 nuc01 java[6482]: 2025-01-23T07:43:19,804 [1062-176] INFO  [ontroller.api.common.ApiWorker] [ctrlApiRest0] Set [ess0/FeedPowerParaSet] to [10000] via API. Timeout is [60s]
Jan 23 07:43:19 nuc01 java[6482]: 2025-01-23T07:43:19,840 [_cycle  ] INFO  [ontroller.api.common.ApiWorker] [ctrlApiRest0] Set Channel [ess0/FeedPowerParaSet] to Value [10000]
Jan 23 07:43:19 nuc01 java[6482]: 2025-01-23T07:43:19,843 [_cycle  ] INFO  [ems.edge.ess.core.power.Solver] Currently active EQUALS constraints
Jan 23 07:43:19 nuc01 java[6482]: 2025-01-23T07:43:19,843 [_cycle  ] INFO  [ems.edge.ess.core.power.Solver] - Channel [SetActivePowerEquals]+ess0P EQUALS 147
Jan 23 07:43:20 nuc01 java[6482]: 2025-01-23T07:43:20,835 [_cycle  ] INFO  [ontroller.api.common.ApiWorker] [ctrlApiRest0] Set Channel [ess0/FeedPowerParaSet] to Value [10000]
Jan 23 07:43:20 nuc01 java[6482]: 2025-01-23T07:43:20,837 [_cycle  ] INFO  [ems.edge.ess.core.power.Solver] Currently active EQUALS constraints
Jan 23 07:43:20 nuc01 java[6482]: 2025-01-23T07:43:20,837 [_cycle  ] INFO  [ems.edge.ess.core.power.Solver] - Channel [SetActivePowerEquals]+ess0P EQUALS 742
Jan 23 07:43:21 nuc01 java[6482]: 2025-01-23T07:43:21,256 [_energy ] ERROR [s.common.worker.AbstractWorker] Worker error. RuntimeException: Error in getting prediction execution - Caused by: java.util.NoSuchElementException
Jan 23 07:43:21 nuc01 java[6482]: java.lang.RuntimeException: Error in getting prediction execution
Jan 23 07:43:21 nuc01 java[6482]:         at io.openems.edge.predictor.lstm.PredictorLstmImpl.createNewPrediction(PredictorLstmImpl.java:162)
Jan 23 07:43:21 nuc01 java[6482]:         at io.openems.edge.predictor.api.prediction.AbstractPredictor.getPrediction(AbstractPredictor.java:68)
Jan 23 07:43:21 nuc01 java[6482]:         at io.openems.edge.core.predictormanager.PredictorManagerImpl.getPrediction(PredictorManagerImpl.java:94)
Jan 23 07:43:21 nuc01 java[6482]:         at io.openems.edge.energy.v1.optimizer.UtilsV1.createSimulatorParams(UtilsV1.java:85)
Jan 23 07:43:21 nuc01 java[6482]:         at io.openems.edge.energy.v1.optimizer.OptimizerV1.createParams(OptimizerV1.java:114)
Jan 23 07:43:21 nuc01 java[6482]:         at io.openems.edge.energy.v1.optimizer.OptimizerV1.forever(OptimizerV1.java:62)
Jan 23 07:43:21 nuc01 java[6482]:         at io.openems.common.worker.AbstractWorker$1.run(AbstractWorker.java:155)
Jan 23 07:43:21 nuc01 java[6482]: Caused by: java.util.concurrent.ExecutionException: java.util.NoSuchElementException
Jan 23 07:43:21 nuc01 java[6482]:         at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
Jan 23 07:43:21 nuc01 java[6482]:         at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
Jan 23 07:43:21 nuc01 java[6482]:         at io.openems.edge.predictor.lstm.PredictorLstmImpl.createNewPrediction(PredictorLstmImpl.java:147)
Jan 23 07:43:21 nuc01 java[6482]:         ... 6 more
Jan 23 07:43:21 nuc01 java[6482]: Caused by: java.util.NoSuchElementException
Jan 23 07:43:21 nuc01 java[6482]:         at java.base/java.util.ArrayList$Itr.next(ArrayList.java:1052)
Jan 23 07:43:21 nuc01 java[6482]:         at java.base/java.util.Collections.min(Collections.java:625)
Jan 23 07:43:21 nuc01 java[6482]:         at io.openems.edge.predictor.lstm.common.HyperParameters.getMinimumErrorModelSeasonality(HyperParameters.java:684)
Jan 23 07:43:21 nuc01 java[6482]:         at io.openems.edge.predictor.lstm.common.HyperParameters.getBestModelSeasonality(HyperParameters.java:766)
Jan 23 07:43:21 nuc01 java[6482]:         at io.openems.edge.predictor.lstm.common.LstmPredictor.predictSeasonality(LstmPredictor.java:42)
Jan 23 07:43:21 nuc01 java[6482]:         at io.openems.edge.predictor.lstm.PredictorLstmImpl.predictSeasonality(PredictorLstmImpl.java:281)
Jan 23 07:43:21 nuc01 java[6482]:         at io.openems.edge.predictor.lstm.PredictorLstmImpl.lambda$createNewPrediction$0(PredictorLstmImpl.java:133)
Jan 23 07:43:21 nuc01 java[6482]:         at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
Jan 23 07:43:21 nuc01 java[6482]:         at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1760)
Jan 23 07:43:21 nuc01 java[6482]:         at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
Jan 23 07:43:21 nuc01 java[6482]:         at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
Jan 23 07:43:21 nuc01 java[6482]:         at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
Jan 23 07:43:21 nuc01 java[6482]:         at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
Jan 23 07:43:21 nuc01 java[6482]:         at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

yvesilknievil avatar Jan 23 '25 07:01 yvesilknievil

Hi @yvesilknievil , could you please share the data file if that's ok?

pooran-c avatar Jan 23 '25 08:01 pooran-c

@pooran-c sure UnmanagedConsumptionActivePower.zip

yvesilknievil avatar Jan 23 '25 09:01 yvesilknievil

Hello

I have tried to activate the LSTM predictor but it has been throwing me errors.

Jan 21 10:40:58 raspberrypi java[34549]: =====> Average RMS error for trend = 299.8913298466389 Jan 21 10:40:58 raspberrypi java[34549]: java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException: Index -1 out of bounds for length 0 Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) Jan 21 10:40:58 raspberrypi java[34549]: at io.openems.edge.predictor.lstm.train.TrainAndValidateBatch.<init>(TrainAndValidateBatch.java:69) Jan 21 10:40:58 raspberrypi java[34549]: at io.openems.edge.predictor.lstm.train.LstmTrain.run(LstmTrain.java:102) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.lang.Thread.run(Thread.java:1583) Jan 21 10:40:58 raspberrypi java[34549]: Caused by: java.lang.IndexOutOfBoundsException: Index -1 out of bounds for length 0 Jan 21 10:40:58 raspberrypi java[34549]: at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:100) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:106) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:302) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.Objects.checkIndex(Objects.java:385) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.ArrayList.get(ArrayList.java:427) Jan 21 10:40:58 raspberrypi java[34549]: at io.openems.edge.predictor.lstm.common.HyperParameters.getlastModelSeasonality(HyperParameters.java:800) Jan 21 10:40:58 raspberrypi java[34549]: at io.openems.edge.predictor.lstm.train.MakeModel.trainSeasonality(MakeModel.java:117) Jan 21 10:40:58 raspberrypi java[34549]: at io.openems.edge.predictor.lstm.train.TrainAndValidateBatch.lambda$new$0(TrainAndValidateBatch.java:53) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1760) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) Jan 21 10:40:58 raspberrypi java[34549]: java.io.FileNotFoundException: /home/user/openems/data/lstm/UnmanagedConsumptionActivePower (No such file or directory) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.io.FileOutputStream.open0(Native Method) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.io.FileOutputStream.open(FileOutputStream.java:289) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:230) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:118) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.io.FileWriter.<init>(FileWriter.java:67) Jan 21 10:40:58 raspberrypi java[34549]: at io.openems.edge.predictor.lstm.common.ReadAndSaveModels.save(ReadAndSaveModels.java:59) Jan 21 10:40:58 raspberrypi java[34549]: at io.openems.edge.predictor.lstm.train.TrainAndValidateBatch.<init>(TrainAndValidateBatch.java:76) Jan 21 10:40:58 raspberrypi java[34549]: at io.openems.edge.predictor.lstm.train.LstmTrain.run(LstmTrain.java:102) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) Jan 21 10:40:58 raspberrypi java[34549]: at java.base/java.lang.Thread.run(Thread.java:1583) Jan 21 10:40:58 raspberrypi java[34549]: =====> Batch = 2/10 Jan 21 10:40:58 raspberrypi java[34549]: =====> Epoch= 0/10

A few questions do rise: I haven't built the code myself but used the jar file. I have declared the correct location of the data file in the service file (ubuntu). Do you need to use rrd4j in order to feed the algorithm with data? I have all my data stored (from 2 years) in influx.

Hi @yvesilknievil. I believe the issue lies in the batchSize variable within the HyperParameters class. This parameter determines how the dataset is divided into smaller batches for training, helping to manage the system’s load. Currently, the batchSize is set to 10 by default.

To address this, please navigate to the file located at io.openems.edge.predictor.lstm/src/io/openems/edge/predictor/lstm/common/HyperParameters.java and modify the batchSize variable to 1. Additionally, delete the file corresponding to the channel ID you are using for training from the ‘Model’ directory before starting the training process. Please let me know if it resolves the issue.

Bishalghimire1997 avatar Jan 23 '25 14:01 Bishalghimire1997

That seems to have done the trick!

Thanks @Bishalghimire1997 and @pooran-c

yvesilknievil avatar Jan 23 '25 19:01 yvesilknievil

24.2.2025, 12:05:48 ERROR io.openems.common.worker.AbstractWorker Worker error. RuntimeException: Error in getting prediction execution - Caused by: java.util.NoSuchElementException

Why could this Error occur?

I changed this:

private static final long DAYS_45 = 45;

to:

private static final long DAYS_15 = 15;

Sn0w3y avatar Feb 24 '25 11:02 Sn0w3y