flow icon indicating copy to clipboard operation
flow copied to clipboard

Collect most frequent Flow build failures in Bender inside this ticket

Open mshabarov opened this issue 4 years ago • 2 comments

Flow builds on the Bender board https://bender.vaadin.com/project.html?projectId=Flow&tab=projectOverview fail sometimes with the following errors:

  • port is already occupied by another process (unfortunately, I couldn't find recent builds with that failure)
  • build execution timeout, however all the tests took <=1-2 secs to run:
[03:31:25]	[Step 5/6] The build Flow 7.0::Flow 7.0 Snapshot #1427 {buildId=253945} has been running for more than 90 minutes. Terminating...
[03:34:34]	[Step 5/6] Execution timeout
[03:31:25]	[Step 5/6] Stopping build on agent. Reason: build execution timeout

Happens for 2.X and 7.0+.

  • huge webpack start time:
Caused by: java.lang.AssertionError: startup time expected <= 12000 but was 230758
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at com.vaadin.flow.testnpmonlyfeatures.performanceregression.StartupPerformanceIT.devModeInitializerToWebpackUpIsBelowThreshold(StartupPerformanceIT.java:60)
  • webpack connection refused (unfortunately, I couldn't find recent builds with that failure)
  • sudden VM exit or crash:
[01:37:06]	[Step 5/6] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.22.2:verify (default) on project flow-test-npm-bytecode-scanning-devmode: There are test failures.

Please refer to /opt/agent/work/67776874a33a15db/flow-tests/test-npm-only-features/test-npm-bytecode-scanning/target/failsafe-reports for the individual test results.
Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
Command was /bin/sh -c cd /opt/agent/work/67776874a33a15db/flow-tests/test-npm-only-features/test-npm-bytecode-scanning && /usr/lib/jvm/jdk1.8.0_201/jre/bin/java -jar /opt/agent/work/67776874a33a15db/flow-tests/test-npm-only-features/test-npm-bytecode-scanning/target/surefire/surefirebooter8191427252273877085.jar /opt/agent/work/67776874a33a15db/flow-tests/test-npm-only-features/test-npm-bytecode-scanning/target/surefire 2021-08-24T22-07-02_151-jvmRun1 surefire4109079052207179689tmp surefire_257294102143676832512tmp
Error occurred in starting fork, check output in log
Process Exit Code: 1
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:669)
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:282)
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:245)
	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1183)
	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1011)
	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:857)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:954)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:192)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)

Investigate the root cause of those error with time-boxed effort - 2 days.

mshabarov avatar Sep 06 '21 09:09 mshabarov

Another JVM crash case:

[09:14:33][Step 7/11] ------------------ Starting Frontend compilation. ------------------
[09:14:36][Step 7/11] (node:24169) [DEP0131] DeprecationWarning: The legacy HTTP parser is deprecated.
[09:14:36][Step 7/11] ? ?wds?: Project is running at http://localhost:35731/
[09:14:36][Step 7/11] ? ?wds?: webpack output is served from /
[09:14:36][Step 7/11] ? ?wds?: Content not from webpack is served from /opt/agent/work/a69251da2b2bb9e9/flow-tests/test-ccdm/target/META-INF/VAADIN/webapp, src/main/webapp
[09:17:28][Step 7/11] #
[09:17:34][Step 7/11] # There is insufficient memory for the Java Runtime Environment to continue.
[09:18:41][Step 7/11] # Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory.
[09:18:53][Step 7/11] # An error report file with more information is saved as:
[09:18:57][Step 7/11] # /opt/agent/work/a69251da2b2bb9e9/hs_err_pid32026.log
[09:18:59][Step 7/11] [thread 140216462149376 also had an error][thread 140213695125248 also had an error][thread 140213714073344 also had an error]
[09:19:00][Step 7/11] 
[09:19:00][Step 7/11] 
[09:19:01][Step 7/11] Publishing artifacts
[09:19:02][Step 7/11] Process exited with code 1 (Step: Run validation (Maven) (Maven))
[09:19:01][Step 7/11] Waiting for 126 service processes to complete
[09:19:01][Step 7/11] Surefire report watcher
[09:19:01][Step 7/11] Surefire report watcher
[09:19:01][Step 7/11] Surefire report watcher
[09:19:01][Step 7/11] Surefire report watcher
[09:19:01][Step 7/11] Surefire report watcher
[09:19:01][Step 7/11] Surefire report watcher
[09:19:01][Step 7/11] Surefire report watcher
[09:19:01][Step 7/11] Surefire report watcher
[09:19:01][Step 7/11] Surefire report watcher
[09:19:01][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Surefire report watcher
[09:19:02][Step 7/11] Found JVM crash log at path: /opt/agent/work/a69251da2b2bb9e9/hs_err_pid32026.log, will publish it as artifact
[09:19:02][Step 7/11] Publishing artifacts
[09:19:02][Step 7/11] JVM crashed
[09:19:02][Step 7/11] Step Run validation (Maven) (Maven) failed

hs_err_pid32026.log.txt

mshabarov avatar Sep 06 '21 10:09 mshabarov