ScreenshotOnFailure doesn't work

[jukzi]

trafficstars

the ScreenshotOnFailure TestWatcher produces only black images on failed tests because the @After org.eclipse.swt.tests.junit.Test_org_eclipse_swt_widgets_Widget.tearDown() calls shell.dispose(); before Screenshot is taken

reproducable with the failing Test_org_eclipse_swt_custom_StyledText

[jukzi]

see https://stackoverflow.com/questions/18008241/run-a-testwatcher-before-the-after https://github.com/junit-team/junit4/issues/906 https://www.thinkcode.se/blog/2012/07/08/performing-an-action-when-a-test-fails

[basilevs]

@jukzi is this reproducible locally? Does it happen on Windows only?

[jukzi]

if i force a test to fail, i can locally reproduce that only the desktop is captured but not the failed junit test (expected to show the wrong color). One the one hand that shows that the ScreenshotOnFailure does not work as intended. On the other hand the dark screen captured on the i build may indicate some screensaver on the server.

[tomaswolf]

@jukzi : we ran into this screenshot taking issue (screenshot taken after the @after have run) in the EGit UI tests back in 2016. Our solution back then was to amend the SwtBotJUnit4ClassRunner to take the screenshot before the @afters. Maybe the things we did in that commit might be useful here, too.

[jukzi]

if some @After/https://github.com/before method fails or a rule such as @ExpectedException fails the test. In those cases, the behavior is unchanged: these screenshots are still taken after @after has run.

@tomaswolf thanks, i like that idea.

[jukzi]

SWTBotJunit4ClassRunner looks a bit like TracingSuite already used in AllBrowserTests. Maybe that TracingSuite could be enhanced to take screenshots also on fail and be used in other suits as well.

[iloveeclipse]

@jukzi : some of the recent changes in this area broke a lot of UI & SWT tests.

Typical stack from https://download.eclipse.org/eclipse/downloads/drops4/I20241013-1800/testresults/html/org.eclipse.swt.tests_ep434I-unit-cen64-gtk3-java17_linux.gtk.x86_64_17.html

Cannot invoke "java.util.function.Supplier.get()" because "this.shell" is null

java.lang.NullPointerException: Cannot invoke "java.util.function.Supplier.get()" because "this.shell" is null
at org.eclipse.test.Screenshots$ScreenshotOnFailure.dispose(Screenshots.java:52)
at org.eclipse.test.Screenshots$ScreenshotOnFailure.finished(Screenshots.java:47)

Could you please fix it?

[jukzi]

Can u please just add a obvious null check. (I am afk)

Jörg Kubitz

Am 14.10.2024 um 10:03 schrieb Andrey Loskutov @.***>:

 @jukzi : some of the recent changes in this area broke a lot of UI & SWT tests.

Typical stack from https://download.eclipse.org/eclipse/downloads/drops4/I20241013-1800/testresults/html/org.eclipse.swt.tests_ep434I-unit-cen64-gtk3-java17_linux.gtk.x86_64_17.html

Cannot invoke "java.util.function.Supplier.get()" because "this.shell" is null

java.lang.NullPointerException: Cannot invoke "java.util.function.Supplier.get()" because "this.shell" is null at org.eclipse.test.Screenshots$ScreenshotOnFailure.dispose(Screenshots.java:52) at org.eclipse.test.Screenshots$ScreenshotOnFailure.finished(Screenshots.java:47) Could you please fix it?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[iloveeclipse]

Can u please just add a obvious null check. (I am afk)

OK, see https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/pull/2452

[jukzi]

Thanks!

Jörg Kubitz

Am 14.10.2024 um 10:38 schrieb Andrey Loskutov @.***>:

 Can u please just add a obvious null check. (I am afk)

OK, see eclipse-platform/eclipse.platform.releng.aggregator#2452

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[jukzi]

The screenshots are still not written. error is: "no screen devices"

Exception in thread "main" java.awt.AWTError: no screen devices
	at java.desktop/sun.awt.Win32GraphicsEnvironment.getDefaultScreenDevice(Win32GraphicsEnvironment.java:101)
	at java.desktop/java.awt.Robot.<init>(Robot.java:98)
	at org.eclipse.test.AwtScreenshot.main(AwtScreenshot.java:41)

https://ci.eclipse.org/releng/job/AutomatedTests/job/ep434I-unit-win32-java17/84/execution/node/3/ws/workarea/I20241026-1800/eclipse-testing/results/ep434I-unit-win32-java17_win32.win32.x86_64_17/org.eclipse.swt.tests.junit.AllNonBrowserTests.txt

[jukzi]

"no screen devices" may relate to https://bugs.openjdk.org/browse/JDK-8336862

[iloveeclipse]

"no screen devices" may relate to https://bugs.openjdk.org/browse/JDK-8336862

Great, isn't that the reason for https://github.com/eclipse-platform/eclipse.platform.swt/issues/1486 ?

So we should downgrade JDK on Windows to 17.0.11 version to get tests working again???

[laeubi]

Too bad we need AWT for SWT tests ...

So probably better use

final Display display = Display.getDefault();
final Image captureImage = new Image(display, display.getBounds().width, display.getBounds().height);
GC gc = new GC(display);
gc.copyArea(captureImage, display.getBounds().x, display.getBounds().y);
gc.dispose();
Imagewriter.write(...)

(see for example https://github.com/eclipse-swtbot/org.eclipse.swtbot/blob/ed2034a65915f6095d3c2464ac43068b105ebd68/org.eclipse.swtbot.swt.finder/src/org/eclipse/swtbot/swt/finder/utils/SWTUtils.java#L463 )

[mjschwaiger]

Update to JDK-8185862: Yesterday a new issue JDK-8348625 was created which proposes to revert the changes made in JDK-8185862 for JDK 11, 17, and 21.