intern icon indicating copy to clipboard operation
intern copied to clipboard

Run multiple `intern.run()` with API

Open AmirTugi opened this issue 6 years ago • 6 comments

The Problem

When trying to run intern more than one or even the same intern more than once, using intern.run(), after re-configuring it, it will shout that it cannot run more than once + due to intern using global scope - cannot run more than 1 in parallel.

This behaviour is restricting the user from running and handling intern as he wishes (in our case, make a grid of intern runners who run parallelly)

Possible solution

When removing the restriction, and maybe make a cleanup (also using other, more isolated scopes for resources) of resources after the intern.run() we could run multiple interns parallelly, and we'll have more freedom and flexability with intern's api

Steps to reproduce (for bugs)

Just try to run the same suites twice, either parallelly, or in series.

Environment

Intern version: 4.2.0

AmirTugi avatar Jun 06 '18 13:06 AmirTugi

The current plan for running Intern in parallel is to use multiple node processes. Intern would start up a separate node process to run tests, much like it starts a browser to run tests now. This will allow for the same logic to manage tests in Node and browsers, and will provide isolation between parallel Node sessions.

jason0x43 avatar Jun 06 '18 13:06 jason0x43

A related issue is that you cannot run parallel functional tests at all. Unless they are configured on different browsers - which means I can't run 2 tests in parallel on chrome v66 for example.

AlonMiz avatar Jun 07 '18 06:06 AlonMiz

Yeah, we have https://github.com/theintern/intern/issues/546 for sharding across multiple instances of a single environment and https://github.com/theintern/intern/issues/803 for running Node as a standard environment. Between the two of those we should end up with a pretty nice parallelization platform.

jason0x43 avatar Jun 07 '18 12:06 jason0x43

@jason0x43 any news? :/ currently, our solution is to spawn multiple child node processes, get the stdout and stderr, and parse the output to figure out fail and success suites and tests. this is really hard and prone to errors, and really ugly.

AlonMiz avatar Jul 31 '18 20:07 AlonMiz

It's high in the queue, but I haven't had time to get to it yet.

In the meantime you might consider writing a custom reporter that will output test results in a structured format, like JSON. That will make result aggregation simpler.

jason0x43 avatar Jul 31 '18 20:07 jason0x43

Bumping this one up as I've recently run into a similar issue. I am using the same technique as @AlonMiz to usable effect, though it is kludgy as suggested.

jacobroufa avatar Jan 29 '19 17:01 jacobroufa