NullPointerException in SendMAEStep when performing indices restoration

[Masterchen09]

Describe the bug After updating to version 0.13.2 the restore indices job is throwing a NullPointerException in class SendMAEStep.

To Reproduce Start the restore indices job after upgrading to version 0.13.2 and check the log - once the job is reaching the SendMAEStep a NullPointerException is thrown. It seems sending the MAEs is still working (somehow) regardless of the NullPointerException.

[main] ERROR c.l.d.u.restoreindices.SendMAEStep:78 - Error iterating futures
java.util.concurrent.ExecutionException: java.lang.NullPointerException: Cannot invoke "io.ebean.bean.PersistenceContext.endIterate()" because "this.persistenceContext" is null
	at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
	at com.linkedin.datahub.upgrade.restoreindices.SendMAEStep.iterateFutures(SendMAEStep.java:75)
	at com.linkedin.datahub.upgrade.restoreindices.SendMAEStep.lambda$executable$0(SendMAEStep.java:198)
	at com.linkedin.datahub.upgrade.impl.DefaultUpgradeManager.executeStepInternal(DefaultUpgradeManager.java:125)
	at com.linkedin.datahub.upgrade.impl.DefaultUpgradeManager.executeInternal(DefaultUpgradeManager.java:76)
	at com.linkedin.datahub.upgrade.impl.DefaultUpgradeManager.executeInternal(DefaultUpgradeManager.java:45)
	at com.linkedin.datahub.upgrade.impl.DefaultUpgradeManager.execute(DefaultUpgradeManager.java:34)
	at com.linkedin.datahub.upgrade.UpgradeCli.run(UpgradeCli.java:86)
	at org.springframework.boot.SpringApplication.lambda$callRunner$5(SpringApplication.java:790)
	at org.springframework.util.function.ThrowingConsumer$1.acceptWithException(ThrowingConsumer.java:83)
	at org.springframework.util.function.ThrowingConsumer.accept(ThrowingConsumer.java:60)
	at org.springframework.util.function.ThrowingConsumer$1.accept(ThrowingConsumer.java:88)
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798)
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:789)
	at org.springframework.boot.SpringApplication.lambda$callRunners$3(SpringApplication.java:774)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:510)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:774)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:341)
	at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:149)
	at com.linkedin.datahub.upgrade.UpgradeCliApplication.main(UpgradeCliApplication.java:42)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:91)
	at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:53)
	at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:58)
Caused by: java.lang.NullPointerException: Cannot invoke "io.ebean.bean.PersistenceContext.endIterate()" because "this.persistenceContext" is null
	at io.ebeaninternal.server.core.OrmQueryRequest.endTransIfRequired(OrmQueryRequest.java:296)
	at io.ebeaninternal.server.core.DefaultServer.findIterate(DefaultServer.java:1374)
	at io.ebeaninternal.server.core.DefaultServer.findStream(DefaultServer.java:1386)
	at io.ebeaninternal.server.querydefn.DefaultOrmQuery.findStream(DefaultOrmQuery.java:1456)
	at com.linkedin.metadata.entity.ebean.EbeanAspectDao.streamAspectBatches(EbeanAspectDao.java:557)
	at com.linkedin.metadata.entity.EntityServiceImpl.streamRestoreIndices(EntityServiceImpl.java:1189)
	at com.linkedin.datahub.upgrade.restoreindices.SendMAEStep$KafkaJob.call(SendMAEStep.java:51)
	at com.linkedin.datahub.upgrade.restoreindices.SendMAEStep$KafkaJob.call(SendMAEStep.java:40)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

Expected behavior No NullPointerException should be thrown. 😉

Screenshots n/a

Desktop (please complete the following information): n/a

Additional context It is possible that the changes in PR #10241 (@david-leifker) could have caused this issue.

[gabrielwry]

We are also seeing this issue after upgrading to 0.13.2 when running the restore-indices job, can someone help? 🆘

[Masterchen09]

The issue seems to be fixed by PR https://github.com/datahub-project/datahub/pull/10506 (tested with the image acryldata/datahub-upgrade:pr10506).

See also the discussion here: https://datahubspace.slack.com/archives/C029A3M079U/p1714461624761899