birt icon indicating copy to clipboard operation
birt copied to clipboard

Strange error "Can not retrieve data to generate the report"

Open kgibm opened this issue 3 years ago • 10 comments

I'm upgrading an old RCP application from Java 8 to Java 11 and as part of that, I'm upgrading the BIRT update site of https://download.eclipse.org/birt/update-site/2018-09-interim/ to https://download.eclipse.org/birt/update-site/4.9.0/

A feature of the app is using BIRT to generate HTML reports. What's really strange is that when I run the app from within Eclipse, this works fine, but when I build it using Tycho and Maven and launch from there, the report is basically blank and has these sorts of exceptions (despite the error suggesting there is no data available, the application GUI shows plenty of data):

org.eclipse.birt.report.engine.api.EngineException: There is an error in the report query loading, Can not retrieve data to generate the report.
Table (id = 288):
- There is an error in the report query loading, Can not retrieve data to generate the report. 

Error.ReportQueryLoadingError ( 1 time(s) )
detail : org.eclipse.birt.report.engine.api.EngineException: There is an error in the report query loading, Can not retrieve data to generate the report. 
	at org.eclipse.birt.report.engine.data.dte.DataPresentationEngine.doExecuteQuery(DataPresentationEngine.java:121)
	at org.eclipse.birt.report.engine.data.dte.AbstractDataEngine.execute(AbstractDataEngine.java:254)
	at org.eclipse.birt.report.engine.executor.ExecutionContext.executeQuery(ExecutionContext.java:1632)
	at org.eclipse.birt.report.engine.internal.document.v4.ReportItemExecutor.executeQuery(ReportItemExecutor.java:369)
	at org.eclipse.birt.report.engine.internal.document.v4.TableItemExecutor.doExecute(TableItemExecutor.java:68)
	at org.eclipse.birt.report.engine.internal.document.v4.ReportItemExecutor.execute(ReportItemExecutor.java:270)
	at org.eclipse.birt.report.engine.internal.document.v4.ContainerExecutor.prepareChildExecutor(ContainerExecutor.java:200)
	at org.eclipse.birt.report.engine.internal.document.v4.ContainerExecutor.hasNextChild(ContainerExecutor.java:108)
	at org.eclipse.birt.report.engine.internal.executor.wrap.WrappedReportItemExecutor.hasNextChild(WrappedReportItemExecutor.java:84)
	at org.eclipse.birt.report.engine.internal.executor.wrap.WrappedReportItemExecutor.hasNextChild(WrappedReportItemExecutor.java:84)
	at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:60)
	at org.eclipse.birt.report.engine.layout.html.HTMLStackingLM.layoutChildren(HTMLStackingLM.java:27)
	at org.eclipse.birt.report.engine.layout.html.HTMLAbstractLM.layout(HTMLAbstractLM.java:133)
	at org.eclipse.birt.report.engine.layout.html.HTMLInlineStackingLM.resumeLayout(HTMLInlineStackingLM.java:97)
	at org.eclipse.birt.report.engine.layout.html.HTMLInlineStackingLM.layoutNodes(HTMLInlineStackingLM.java:136)
	at org.eclipse.birt.report.engine.layout.html.HTMLStackingLM.layoutChildren(HTMLStackingLM.java:27)
	at org.eclipse.birt.report.engine.layout.html.HTMLAbstractLM.layout(HTMLAbstractLM.java:133)
	at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:66)
	at org.eclipse.birt.report.engine.layout.html.HTMLStackingLM.layoutChildren(HTMLStackingLM.java:27)
	at org.eclipse.birt.report.engine.layout.html.HTMLRepeatHeaderLM.layoutChildren(HTMLRepeatHeaderLM.java:54)
	at org.eclipse.birt.report.engine.layout.html.HTMLAbstractLM.layout(HTMLAbstractLM.java:133)
	at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:66)
	at org.eclipse.birt.report.engine.layout.html.HTMLPageLM.layout(HTMLPageLM.java:92)
	at org.eclipse.birt.report.engine.layout.html.HTMLReportLayoutEngine.layout(HTMLReportLayoutEngine.java:97)
	at org.eclipse.birt.report.engine.api.impl.RenderTask$PageRangeRender.render(RenderTask.java:569)
	at org.eclipse.birt.report.engine.api.impl.RenderTask.render(RenderTask.java:273)
	at com.ibm.perf.ui.editor.ReportEditor.generateHTML(ReportEditor.java:153)
	at com.ibm.perf.ui.editor.ReportEditor.runReport(ReportEditor.java:109)
	at com.ibm.perf.ui.Controller$8$1.run(Controller.java:792)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:132)
	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4345)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3968)
	at com.ibm.perf.ui.ImageProgress.run(ImageProgress.java:111)
	at com.ibm.perf.ui.ImageProgress.start(ImageProgress.java:73)
	at com.ibm.perf.ui.Controller.OpenReportPage(Controller.java:809)
	at com.ibm.perf.ui.handlers.OpenReportPageHandler.execute(OpenReportPageHandler.java:13)
	at org.eclipse.ui.internal.handlers.HandlerProxy.execute(HandlerProxy.java:283)
	at org.eclipse.ui.internal.handlers.E4HandlerProxy.execute(E4HandlerProxy.java:97)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.eclipse.e4.core.internal.di.MethodRequestor.execute(MethodRequestor.java:58)
	at org.eclipse.e4.core.internal.di.InjectorImpl.invokeUsingClass(InjectorImpl.java:317)
	at org.eclipse.e4.core.internal.di.InjectorImpl.invoke(InjectorImpl.java:251)
	at org.eclipse.e4.core.contexts.ContextInjectionFactory.invoke(ContextInjectionFactory.java:173)
	at org.eclipse.e4.core.commands.internal.HandlerServiceHandler.execute(HandlerServiceHandler.java:156)
	at org.eclipse.core.commands.Command.executeWithChecks(Command.java:488)
	at org.eclipse.core.commands.ParameterizedCommand.executeWithChecks(ParameterizedCommand.java:485)
	at org.eclipse.e4.core.commands.internal.HandlerServiceImpl.executeHandler(HandlerServiceImpl.java:213)
	at org.eclipse.e4.ui.workbench.renderers.swt.HandledContributionItem.executeItem(HandledContributionItem.java:438)
	at org.eclipse.e4.ui.workbench.renderers.swt.AbstractContributionItem.handleWidgetSelection(AbstractContributionItem.java:449)
	at org.eclipse.e4.ui.workbench.renderers.swt.AbstractContributionItem.lambda$2(AbstractContributionItem.java:475)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:89)
	at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4604)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1524)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1547)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1532)
	at org.eclipse.swt.widgets.Widget.notifyListeners(Widget.java:1325)
	at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4390)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3966)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$5.run(PartRenderingEngine.java:1155)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1046)
	at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:155)
	at org.eclipse.ui.internal.Workbench.lambda$3(Workbench.java:644)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:551)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:156)
	at com.ibm.perf.ui.Application.start(Application.java:43)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:136)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:402)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:659)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:596)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1467)

There are no obvious differences or exceptions in the System.out/System.err of the two executions leading up to the report generation.

Comparing the installed plugins shows very minor differences (red is the failing app and green is the working app):

$ diff fails_plugins.txt works_plugins.txt 
8c8
< com.ibm.ptt.plugin (0.1.0.202206291830) "Bundle" [Active]
---
> com.ibm.ptt.plugin (0.1.0.qualifier) "Bundle" [Active]
33c33
< org.apache.batik.dom.birt.ext (1.14.0.v202203150031) "Batik DOM Extension" [Resolved]
---
> org.apache.batik.dom.birt.ext (1.14.0.v202203150031) "Batik DOM Extension" [Installed]
35d34
< org.apache.batik.ext (1.14.0.v20210324-0332) "org.apache.xmlgraphics:batik-ext" [Resolved]
77c76
< org.eclipse.birt.data.aggregation (4.9.0.v202203150031) "Build-in Aggregation Plug-in" [Starting]
---
> org.eclipse.birt.data.aggregation (4.9.0.v202203150031) "Build-in Aggregation Plug-in" [Active]
154,155d152
< org.eclipse.equinox.launcher (1.6.400.v20210924-0641) "Equinox Launcher" [Resolved]
< org.eclipse.equinox.launcher.cocoa.macosx.x86_64 (1.2.500.v20220509-0833) "Equinox Launcher MacOSX Cocoa Fragment" [Resolved]
230c227
< org.eclipse.ui.console (3.11.200.v20220324-0630) "Console" [Starting]
---
> org.eclipse.ui.console (3.11.200.v20220324-0630) "Console" [Active]
261c258
< org.mozilla.javascript (1.7.10.v20190430-1943) "Mozilla Rhino" [Resolved]
---
> org.mozilla.javascript (1.7.5.v201504281450) "Mozilla Rhino" [Resolved]

The basic code to generate the report is:

String reportfile = Controller.toPluginFilePath("reports/perfReport.rpt");
EngineConfig config = new EngineConfig();
HashMap hm = config.getAppContext();
hm.put(EngineConstants.APPCONTEXT_CLASSLOADER_KEY, getClass().getClassLoader());
config.setAppContext(hm);
ReportEngine engine = new ReportEngine(config);
Platform.startup(config);
IReportRunnable report = engine.openReportDesign(reportfile);
SimpleDateFormat sdf = new SimpleDateFormat("yyyyMMdd_HHmmss");
Date nameDate = new Date();
if (record.getStartDate() != null) {
    nameDate = record.getStartDate();
} else if (record.getEndDate() != null) {
    nameDate = record.getEndDate();
}
String dtstr = sdf.format(nameDate);
File rptdocument = new File("ptt_report_" + dtstr + ".rptdocument");
IRunTask task = engine.createRunTask(report);
task.run(rptdocument.getAbsolutePath());
task.close();
IReportDocument iReportDocument = engine.openReportDocument(rptdocument.getAbsolutePath());
IRenderTask renderTask = engine.createRenderTask(iReportDocument);
File htmlfile = new File("ptt_report_" + dtstr + ".html");
HTMLRenderOption options = new HTMLRenderOption();
options.setOutputFileName(htmlfile.getAbsolutePath());
options.setOutputFormat("html");
renderTask.setRenderOption(options);
renderTask.render();
renderTask.close();

Any thoughts?

I can't share the whole app for you to reproduce but I can share snippets or try different debug patches.

Attached are the failing and working inputs and outputs in a zip.

engine_exception.zip

kgibm avatar Jun 29 '22 19:06 kgibm

I was looking at the report source and it included lines like this which are JavaScript:

<expression name="expression" type="javascript">dataSetRow["Server"]</expression>

So given it was having issues resolving data and given the plugin diff showed a version difference in the org.mozilla.javascript plugin, I explicitly set the version in my Eclipse .product file and re-ran the Maven build and re-tested:

<plugin id="org.mozilla.javascript" version="1.7.5.v201504281450"/>

That fixed it! It seems something between versions 1.7.5.v201504281450 and 1.7.10.v20190430-1943 is causing the org.eclipse.birt.report.engine.api.EngineException.

kgibm avatar Jun 29 '22 20:06 kgibm

Thanks, Kevin! Can you try to isolate the issue so that it can be fixed for the newer version of mozilla.javascript?

wimjongman avatar Jun 30 '22 10:06 wimjongman

Thanks, Kevin! Can you try to isolate the issue so that it can be fixed for the newer version of mozilla.javascript?

Nothing changed in my app... do you mean trying something like git bisect to try to find the version the behavior of org.mozilla.javascript changed to cause the issue?

kgibm avatar Jun 30 '22 13:06 kgibm

I mean where does BIRT fail using the newer library?

wimjongman avatar Jul 04 '22 14:07 wimjongman

It's in the bug description:

org.eclipse.birt.report.engine.api.EngineException: There is an error in the report query loading, Can not retrieve data to generate the report.
Table (id = 288):
- There is an error in the report query loading, Can not retrieve data to generate the report. 

Error.ReportQueryLoadingError ( 1 time(s) )
detail : org.eclipse.birt.report.engine.api.EngineException: There is an error in the report query loading, Can not retrieve data to generate the report. 
	at org.eclipse.birt.report.engine.data.dte.DataPresentationEngine.doExecuteQuery(DataPresentationEngine.java:121)
	at org.eclipse.birt.report.engine.data.dte.AbstractDataEngine.execute(AbstractDataEngine.java:254)
	at org.eclipse.birt.report.engine.executor.ExecutionContext.executeQuery(ExecutionContext.java:1632)
	at org.eclipse.birt.report.engine.internal.document.v4.ReportItemExecutor.executeQuery(ReportItemExecutor.java:369)
	at org.eclipse.birt.report.engine.internal.document.v4.TableItemExecutor.doExecute(TableItemExecutor.java:68)
	at org.eclipse.birt.report.engine.internal.document.v4.ReportItemExecutor.execute(ReportItemExecutor.java:270)
	at org.eclipse.birt.report.engine.internal.document.v4.ContainerExecutor.prepareChildExecutor(ContainerExecutor.java:200)
	at org.eclipse.birt.report.engine.internal.document.v4.ContainerExecutor.hasNextChild(ContainerExecutor.java:108)
	at org.eclipse.birt.report.engine.internal.executor.wrap.WrappedReportItemExecutor.hasNextChild(WrappedReportItemExecutor.java:84)
	at org.eclipse.birt.report.engine.internal.executor.wrap.WrappedReportItemExecutor.hasNextChild(WrappedReportItemExecutor.java:84)
	at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:60)
	at org.eclipse.birt.report.engine.layout.html.HTMLStackingLM.layoutChildren(HTMLStackingLM.java:27)
	at org.eclipse.birt.report.engine.layout.html.HTMLAbstractLM.layout(HTMLAbstractLM.java:133)
	at org.eclipse.birt.report.engine.layout.html.HTMLInlineStackingLM.resumeLayout(HTMLInlineStackingLM.java:97)
	at org.eclipse.birt.report.engine.layout.html.HTMLInlineStackingLM.layoutNodes(HTMLInlineStackingLM.java:136)
	at org.eclipse.birt.report.engine.layout.html.HTMLStackingLM.layoutChildren(HTMLStackingLM.java:27)
	at org.eclipse.birt.report.engine.layout.html.HTMLAbstractLM.layout(HTMLAbstractLM.java:133)
	at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:66)
	at org.eclipse.birt.report.engine.layout.html.HTMLStackingLM.layoutChildren(HTMLStackingLM.java:27)
	at org.eclipse.birt.report.engine.layout.html.HTMLRepeatHeaderLM.layoutChildren(HTMLRepeatHeaderLM.java:54)
	at org.eclipse.birt.report.engine.layout.html.HTMLAbstractLM.layout(HTMLAbstractLM.java:133)
	at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:66)
	at org.eclipse.birt.report.engine.layout.html.HTMLPageLM.layout(HTMLPageLM.java:92)
	at org.eclipse.birt.report.engine.layout.html.HTMLReportLayoutEngine.layout(HTMLReportLayoutEngine.java:97)
	at org.eclipse.birt.report.engine.api.impl.RenderTask$PageRangeRender.render(RenderTask.java:569)
	at org.eclipse.birt.report.engine.api.impl.RenderTask.render(RenderTask.java:273)
	at com.ibm.perf.ui.editor.ReportEditor.generateHTML(ReportEditor.java:153)
	at com.ibm.perf.ui.editor.ReportEditor.runReport(ReportEditor.java:109)
	at com.ibm.perf.ui.Controller$8$1.run(Controller.java:792)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:132)
	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4345)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3968)
	at com.ibm.perf.ui.ImageProgress.run(ImageProgress.java:111)
	at com.ibm.perf.ui.ImageProgress.start(ImageProgress.java:73)
	at com.ibm.perf.ui.Controller.OpenReportPage(Controller.java:809)
	at com.ibm.perf.ui.handlers.OpenReportPageHandler.execute(OpenReportPageHandler.java:13)
	at org.eclipse.ui.internal.handlers.HandlerProxy.execute(HandlerProxy.java:283)
	at org.eclipse.ui.internal.handlers.E4HandlerProxy.execute(E4HandlerProxy.java:97)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.eclipse.e4.core.internal.di.MethodRequestor.execute(MethodRequestor.java:58)
	at org.eclipse.e4.core.internal.di.InjectorImpl.invokeUsingClass(InjectorImpl.java:317)
	at org.eclipse.e4.core.internal.di.InjectorImpl.invoke(InjectorImpl.java:251)
	at org.eclipse.e4.core.contexts.ContextInjectionFactory.invoke(ContextInjectionFactory.java:173)
	at org.eclipse.e4.core.commands.internal.HandlerServiceHandler.execute(HandlerServiceHandler.java:156)
	at org.eclipse.core.commands.Command.executeWithChecks(Command.java:488)
	at org.eclipse.core.commands.ParameterizedCommand.executeWithChecks(ParameterizedCommand.java:485)
	at org.eclipse.e4.core.commands.internal.HandlerServiceImpl.executeHandler(HandlerServiceImpl.java:213)
	at org.eclipse.e4.ui.workbench.renderers.swt.HandledContributionItem.executeItem(HandledContributionItem.java:438)
	at org.eclipse.e4.ui.workbench.renderers.swt.AbstractContributionItem.handleWidgetSelection(AbstractContributionItem.java:449)
	at org.eclipse.e4.ui.workbench.renderers.swt.AbstractContributionItem.lambda$2(AbstractContributionItem.java:475)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:89)
	at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4604)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1524)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1547)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1532)
	at org.eclipse.swt.widgets.Widget.notifyListeners(Widget.java:1325)
	at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4390)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3966)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$5.run(PartRenderingEngine.java:1155)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1046)
	at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:155)
	at org.eclipse.ui.internal.Workbench.lambda$3(Workbench.java:644)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:551)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:156)
	at com.ibm.perf.ui.Application.start(Application.java:43)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:136)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:402)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:659)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:596)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1467)

kgibm avatar Jul 04 '22 15:07 kgibm

I noticed that your report is using script data sets with an event handler "com.ibm.perf.ui.handlers.ScriptDataSetHandler". Can you reproduce the issue with a JDBC data source and JDBC data sets as well? Maybe the problem is related to the event handler...

hvbtup avatar Jul 05 '22 06:07 hvbtup

I noticed that your report is using script data sets with an event handler "com.ibm.perf.ui.handlers.ScriptDataSetHandler".

Yes, it implements org.eclipse.birt.report.engine.api.script.eventhandler.IScriptedDataSetEventHandler

I took over this code from someone that left my company, so I'm not very familiar with it. There's nothing complicated or proprietary in ScriptDataSetHandler.java so I'm sharing it here in case it gives any hints about the problem:

ScriptDataSetHandler.java
package com.ibm.perf.ui.handlers;

import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;

import org.eclipse.birt.report.engine.api.script.IColumnMetaData;
import org.eclipse.birt.report.engine.api.script.IDataSetRow;
import org.eclipse.birt.report.engine.api.script.IReportContext;
import org.eclipse.birt.report.engine.api.script.IScriptedDataSetMetaData;
import org.eclipse.birt.report.engine.api.script.IUpdatableDataSetRow;
import org.eclipse.birt.report.engine.api.script.ScriptException;
import org.eclipse.birt.report.engine.api.script.eventhandler.IScriptedDataSetEventHandler;
import org.eclipse.birt.report.engine.api.script.instance.IDataSetInstance;

import com.ibm.perf.event.ConnPoolEvent;
import com.ibm.perf.event.EjbEvent;
import com.ibm.perf.event.JcaPoolEvent;
import com.ibm.perf.event.ReportData;
import com.ibm.perf.event.ServletEvent;
import com.ibm.perf.event.SessionEvent;
import com.ibm.perf.event.ThreadPoolEvent;

public class ScriptDataSetHandler implements IScriptedDataSetEventHandler {

	protected IColumnMetaData meta = null;
	protected Iterator it = null;
	public static List<ReportData> list = null;

	@Override
	public void beforeOpen(IDataSetInstance dataSet, IReportContext reportContext) throws ScriptException {
		// TODO Auto-generated method stub

	}

	@Override
	public void afterOpen(IDataSetInstance dataSet, IReportContext reportContext) throws ScriptException {

	}

	@Override
	public void onFetch(IDataSetInstance dataSet, IDataSetRow row, IReportContext reportContext)
			throws ScriptException {
		// TODO Auto-generated method stub

	}

	@Override
	public void beforeClose(IDataSetInstance dataSet, IReportContext reportContext) throws ScriptException {
		// TODO Auto-generated method stub

	}

	@Override
	public void afterClose(IReportContext reportContext) throws ScriptException {
		// TODO Auto-generated method stub

	}

	@Override
	public void open(IDataSetInstance dataSet) throws ScriptException {
		try {
			meta = dataSet.getColumnMetaData();
			if (dataSet.getName().equalsIgnoreCase("ReportDataSet")) {
				it = list.iterator();
			} else if (dataSet.getName().equalsIgnoreCase("ServletDS")) {
				ArrayList al = new ArrayList();
				for (ReportData data : list) {
					if (data.getServletList() != null) {
						al.addAll(data.getServletList());
					}
				}
				it = al.iterator();
			} else if (dataSet.getName().equalsIgnoreCase("EJBDataSet")) {
				ArrayList al = new ArrayList();
				for (ReportData data : list) {
					if (data.getEjbList() != null) {
						al.addAll(data.getEjbList());
					}
				}
				it = al.iterator();
			} else if (dataSet.getName().equalsIgnoreCase("JDBCDataSet")) {
				ArrayList al = new ArrayList();
				for (ReportData data : list) {
					if (data.getConnPoolList() != null) {
						al.addAll(data.getConnPoolList());
					}
				}
				it = al.iterator();
			} else if (dataSet.getName().equalsIgnoreCase("JcaDataSet")) {
				ArrayList al = new ArrayList();
				for (ReportData data : list) {
					if (data.getJcaPoolList() != null) {
						al.addAll(data.getJcaPoolList());
					}
				}
				it = al.iterator();
			} else if (dataSet.getName().equalsIgnoreCase("SessionDataSet")) {
				ArrayList al = new ArrayList();
				for (ReportData data : list) {
					if (data.getSessionList() != null) {
						al.addAll(data.getSessionList());
					}
				}
				it = al.iterator();
			} else if (dataSet.getName().equalsIgnoreCase("ThreadPoolDataSet")) {
				ArrayList al = new ArrayList();
				for (ReportData data : list) {
					if (data.getThreadPoolList() != null) {
						al.addAll(data.getThreadPoolList());
					}
				}
				it = al.iterator();
			}

		} catch (Exception e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
	}

	@Override
	public boolean fetch(IDataSetInstance dataSet, IUpdatableDataSetRow row) throws ScriptException {
		if (dataSet.getName().equalsIgnoreCase("ReportDataSet")) {
			if (it == null || !it.hasNext()) {
				return false;
			} else {
				try {
					ReportData data = (ReportData) it.next();
					for (int i = 0; i < meta.getColumnCount(); i++) {
						String colData = "";
						switch (i) {
						case 0:
							colData = String.valueOf(data.getServer());
							break;
						case 1:
							colData = String.valueOf(data.getServerType());
							break;
						case 2:
							colData = String.valueOf(data.getServletError());
							break;
						case 3:
							colData = String.valueOf(data.getTransactionRollback());
							break;
						case 4:
							colData = String.valueOf(data.getJdbcConnectionTimeout());
							break;
						case 5:
							colData = String.valueOf(data.getThreadHang());
							break;
						case 6:
							colData = round(data.getResponseTime());
							break;
						case 7:
							colData = round(data.getThroughput());
							break;
						case 8:
							colData = String.valueOf(data.getCpuUsage());
							break;
						case 9:
							colData = String.valueOf(data.getUsedHeap());
							break;
						case 10:
							colData = String.valueOf(data.getCacheHit());
							break;
						case 11:
							colData = String.valueOf(data.getCacheHitCountValidated());
							break;
						case 12:
							colData = String.valueOf(data.getFailedRequest());
							break;
						case 13:
							colData = String.valueOf(data.getNode());
							break;
						case 14:
							colData = String.valueOf(data.getTransactionEvent().getRolledbackPercentage());
							break;
						case 15:
							colData = String.valueOf(data.getTransactionEvent().getLocalTimeoutCount());
							break;
						case 16:
							colData = String.valueOf(data.getTransactionEvent().getGlobalTimeoutCount());
							break;
						case 17:
							colData = String.valueOf(data.getTransactionEvent().getAvgResponseTime());
							break;
						case 18:
							colData = String.valueOf(data.getTransactionEvent().getThroughput());
							break;
						}
						String colName = meta.getColumnName(i + 1);
						row.setColumnValue(colName, colData);
					}
				} catch (ScriptException e) {
					e.printStackTrace();
				}
				return true;
			}
		} else if (dataSet.getName().equalsIgnoreCase("ServletDS")) {
			if (it == null || !it.hasNext()) {
				return false;
			} else {
				try {
					ServletEvent data = (ServletEvent) it.next();
					for (int i = 0; i < meta.getColumnCount(); i++) {
						String colData = "";
						switch (i) {
						case 0:
							colData = String.valueOf(data.getNode());
							break;
						case 1:
							colData = String.valueOf(data.getServer());
							break;
						case 2:
							colData = String.valueOf(data.getAppName());
							break;
						case 3:
							colData = String.valueOf(data.getName());
							break;
						case 4:
							colData = String.valueOf(data.getErrorCount());
							break;
						case 5:
							colData = String.valueOf(data.getAvgResponseTime());
							break;
						case 6:
							colData = String.valueOf(data.getThroughput());
							break;
						}
						String colName = meta.getColumnName(i + 1);
						row.setColumnValue(colName, colData);
					}
				} catch (ScriptException e) {
					e.printStackTrace();
				}
				return true;
			}
		} else if (dataSet.getName().equalsIgnoreCase("EJBDataSet")) {
			if (it == null || !it.hasNext()) {
				return false;
			} else {
				try {
					EjbEvent data = (EjbEvent) it.next();
					for (int i = 0; i < meta.getColumnCount(); i++) {
						String colData = "";
						switch (i) {
						case 0:
							colData = String.valueOf(data.getNode());
							break;
						case 1:
							colData = String.valueOf(data.getServer());
							break;
						case 2:
							colData = String.valueOf(data.getJarName());
							break;
						case 3:
							colData = String.valueOf(data.getBeanType());
							break;
						case 4:
							colData = String.valueOf(data.getBeanName());
							break;
						case 5:
							colData = String.valueOf(data.getMethodName());
							break;
						case 6:
							colData = String.valueOf(data.getAvgResponseTime());
							break;
						case 7:
							colData = String.valueOf(data.getThroughput());
							break;
						}
						String colName = meta.getColumnName(i + 1);
						row.setColumnValue(colName, colData);
					}
				} catch (ScriptException e) {
					e.printStackTrace();
				}
				return true;
			}
		} else if (dataSet.getName().equalsIgnoreCase("JDBCDataSet")) {
			if (it == null || !it.hasNext()) {
				return false;
			} else {
				try {
					ConnPoolEvent data = (ConnPoolEvent) it.next();
					for (int i = 0; i < meta.getColumnCount(); i++) {
						String colData = "";
						switch (i) {
						case 0:
							colData = String.valueOf(data.getNode());
							break;
						case 1:
							colData = String.valueOf(data.getServer());
							break;
						case 2:
							colData = String.valueOf(data.getName());
							break;
						case 3:
							colData = String.valueOf(data.getAvgWaitTime());
							break;
						case 4:
							colData = String.valueOf(data.getErrorCount());
							break;
						case 5:
							colData = String.valueOf(data.getPrepStmtCacheDiscardCount());
							break;
						case 6:
							colData = String.valueOf(data.getAvgResponseTime());
							break;
						case 7:
							colData = String.valueOf(data.getThroughput());
							break;
						case 8:
							colData = String.valueOf(data.getAvgJdbcResponseTime());
							break;
						case 9:
							colData = String.valueOf(data.getJdbcThroughput());
							break;
						}
						String colName = meta.getColumnName(i + 1);
						row.setColumnValue(colName, colData);
					}
				} catch (ScriptException e) {
					e.printStackTrace();
				}
				return true;
			}
		} else if (dataSet.getName().equalsIgnoreCase("JcaDataSet")) {
			if (it == null || !it.hasNext()) {
				return false;
			} else {
				try {
					JcaPoolEvent data = (JcaPoolEvent) it.next();
					for (int i = 0; i < meta.getColumnCount(); i++) {
						String colData = "";
						switch (i) {
						case 0:
							colData = String.valueOf(data.getNode());
							break;
						case 1:
							colData = String.valueOf(data.getServer());
							break;
						case 2:
							colData = String.valueOf(data.getName());
							break;
						case 3:
							colData = String.valueOf(data.getAvgWaitTime());
							break;
						case 4:
							colData = String.valueOf(data.getErrorCount());
							break;
						case 5:
							colData = String.valueOf(data.getMaxUsagePercentage());
							break;
						case 6:
							colData = String.valueOf(data.getAvgResponseTime());
							break;
						case 7:
							colData = String.valueOf(data.getThroughput());
							break;
						}
						String colName = meta.getColumnName(i + 1);
						row.setColumnValue(colName, colData);
					}
				} catch (ScriptException e) {
					e.printStackTrace();
				}
				return true;
			}
		} else if (dataSet.getName().equalsIgnoreCase("SessionDataSet")) {
			if (it == null || !it.hasNext()) {
				return false;
			} else {
				try {
					SessionEvent data = (SessionEvent) it.next();
					for (int i = 0; i < meta.getColumnCount(); i++) {
						String colData = "";
						switch (i) {
						case 0:
							colData = String.valueOf(data.getNode());
							break;
						case 1:
							colData = String.valueOf(data.getServer());
							break;
						case 2:
							colData = String.valueOf(data.getName());
							break;
						case 3:
							colData = String.valueOf(data.getAffinityBreakCount());
							break;
						case 4:
							colData = String.valueOf(data.getTimeoutInvalidationCount());
							break;
						case 5:
							colData = String.valueOf(data.getLiveCount());
							break;
						case 6:
							colData = String.valueOf(data.getAvgLifeTime());
							break;
						}
						String colName = meta.getColumnName(i + 1);
						row.setColumnValue(colName, colData);
					}
				} catch (ScriptException e) {
					e.printStackTrace();
				}
				return true;
			}
		} else if (dataSet.getName().equalsIgnoreCase("ThreadPoolDataSet")) {
			if (it == null || !it.hasNext()) {
				return false;
			} else {
				try {
					ThreadPoolEvent data = (ThreadPoolEvent) it.next();
					for (int i = 0; i < meta.getColumnCount(); i++) {
						String colData = "";
						switch (i) {
						case 0:
							colData = String.valueOf(data.getNode());
							break;
						case 1:
							colData = String.valueOf(data.getServer());
							break;
						case 2:
							colData = String.valueOf(data.getPoolName());
							break;
						case 3:
							colData = String.valueOf(data.getDeclaredThreadHungCount());
							break;
						case 4:
							colData = String.valueOf(data.getClearedThreadHangCount());
							break;
						case 5:
							colData = String.valueOf(data.getMaxSize());
							break;
						case 6:
							colData = String.valueOf(data.getMinSize());
							break;
						case 7:
							colData = String.valueOf(data.getPoolSize());
							break;
						case 8:
							colData = String.valueOf(data.getPoolUsed());
							break;
						}
						String colName = meta.getColumnName(i + 1);
						row.setColumnValue(colName, colData);
					}
				} catch (ScriptException e) {
					e.printStackTrace();
				}
				return true;
			}
		}
		System.out.println(dataSet.getName() + " is not found!");
		return false;
	}

	@Override
	public void close(IDataSetInstance dataSet) throws ScriptException {
		// TODO Auto-generated method stub

	}

	@Override
	public boolean describe(IDataSetInstance dataSet, IScriptedDataSetMetaData metaData) throws ScriptException {
		// TODO Auto-generated method stub
		return false;
	}

	public static String round(double v) {
		if (v < 0 || Double.isNaN(v)) {
			return String.valueOf(v);
		}
		String temp = "#0.000";
		return new java.text.DecimalFormat(temp).format(v);
	}
}

Can you reproduce the issue with a JDBC data source and JDBC data sets as well? Maybe the problem is related to the event handler...

That seems like a pretty significant change to test. I'll add it to the list to test after trying to binary search on the version of org.mozilla.javascript that's failing.

kgibm avatar Jul 05 '22 13:07 kgibm

Well, at least the exception handling of that code looks really buggy to me.

Most error situations are ignored silently. Lines like System.out.println(...) and e.printStackTrace(); won't have any effect unless you setup your system to actually show System.out and System.err somehow.

I recommend to rewrite the error and exception handling from scratch. A first step to see what's going on might involve removing the exception handling complete or to throw a ScriptException with the current exception as cause.

What I mean is: I think that the terrible exception handling hides the actual error, that's why the stack trace doesn't contain anything useful.

hvbtup avatar Jul 05 '22 13:07 hvbtup

I replaced the printStackTrace calls with throw new ScriptException(e); and I'm running from the command line with -debug -consoleLog so I do see other System.*.println calls, and I reverted to the newer org.mozilla.javascript plugin, but it still fails in the same way (no different ScriptExceptions).

kgibm avatar Jul 05 '22 14:07 kgibm

You could add some additional logging to the Java code to see if your handlers are working as expected (and maybe compare the log output when using the newer javascript plugin to that when using the older plugin.

Anyway, my guess is that root cause of the issue is some unintended behavior change within the Rhino engine.

One question that you didn't answer yet: Can you reproduce the problem with a JDBC based report (without scripted datasets?).

BTW: Did you already search for known issues in the Rhino project that might be related?

I fear that it will boil down to a diff of the Rhino source code of the two different plugin versions.

hvbtup avatar Jul 05 '22 14:07 hvbtup