Policy Based Data Retention ERROR

[stroomdev10]

Policy Based Data Retention - Error enforcing data retention policies: from 2023-08-27T00:00:07.927Z is after to 2023-08-27T00:00:03.049Z 
java.lang.RuntimeException: from 2023-08-27T00:00:07.927Z is after to 2023-08-27T00:00:03.049Z
	at stroom.util.time.TimePeriod.<init>(TimePeriod.java:32)
	at stroom.util.time.TimePeriod.between(TimePeriod.java:43)
	at stroom.data.retention.impl.DataRetentionPolicyExecutor.getProcessPeriods(DataRetentionPolicyExecutor.java:386)
	at stroom.data.retention.impl.DataRetentionPolicyExecutor.process(DataRetentionPolicyExecutor.java:151)
	at stroom.data.retention.impl.DataRetentionPolicyExecutor.lambda$exec$2(DataRetentionPolicyExecutor.java:131)
	at stroom.cluster.lock.impl.db.ClusterLockServiceImpl.tryLock(ClusterLockServiceImpl.java:97)
	at stroom.data.retention.impl.DataRetentionPolicyExecutor.exec(DataRetentionPolicyExecutor.java:124)
	at stroom.data.retention.impl.DataRetentionPolicyExecutor.exec(DataRetentionPolicyExecutor.java:117)
	at stroom.util.RunnableWrapper.run(RunnableWrapper.java:19)
	at stroom.job.impl.ScheduledJobFunction.run(ScheduledJobFunction.java:27)
	at stroom.job.impl.ScheduledTaskExecutor$JobNodeTrackedFunction.run(ScheduledTaskExecutor.java:272)
	at stroom.job.impl.ScheduledTaskExecutor.lambda$execute$2(ScheduledTaskExecutor.java:151)
	at stroom.task.impl.TaskContextFactoryImpl.lambda$createFromConsumer$0(TaskContextFactoryImpl.java:193)
	at stroom.task.impl.TaskContextFactoryImpl.lambda$wrap$2(TaskContextFactoryImpl.java:266)
	at stroom.util.logging.LocationAwareLambdaLogger.logDurationIfDebugEnabled(LocationAwareLambdaLogger.java:149)
	at stroom.task.impl.TaskContextFactoryImpl.lambda$wrap$4(TaskContextFactoryImpl.java:266)
	at stroom.util.pipeline.scope.PipelineScopeRunnable.scopeResult(PipelineScopeRunnable.java:38)
	at stroom.task.impl.TaskContextFactoryImpl.lambda$wrap$5(TaskContextFactoryImpl.java:263)
	at stroom.task.impl.TaskContextFactoryImpl.lambda$wrap$7(TaskContextFactoryImpl.java:277)
	at stroom.security.impl.SecurityContextImpl.asUserResult(SecurityContextImpl.java:285)
	at jdk.internal.reflect.GeneratedMethodAccessor162.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:50)
	at jdk.proxy2/jdk.proxy2.$Proxy116.asUserResult(Unknown Source)
	at stroom.task.impl.TaskContextFactoryImpl.lambda$wrap$11(TaskContextFactoryImpl.java:273)
	at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

[at055612]

It is possible this is caused by clock drift on the server as the server in question was not linked to an NTP server.

[at055612]

The data retention job does a lot of date calculations and they mostly all hang off now() and the tracker times (which came from now() on the prev run). now() currently comes from System.currentTimeInMillis on the host running the job.

It could be changed so that now() comes from asking the db for what it knows to be the current time. Then we have a consistent now time regardless of which server runs the job, which would allow for nodes to be on different times.