Headless Chrome times out without executing any tests

[applecool]

I have configured the headless chrome in my project getting rid of Phantom. When I run the tests on my OSX machine, all the tests get executed with the test runner on the headless chrome. But the same doesn't happen on my linux server. It connects to the socket and then after some time, it disconnects and shows the error message saying "Karma tests failed". Error:

25 09 2017 13:56:19.075:INFO [karma]: Karma v1.5.0 server started at http://0.0.0.0:9876/build/
25 09 2017 13:56:19.076:INFO [launcher]: Launching browser ChromeHeadless with unlimited concurrency
25 09 2017 13:56:19.080:INFO [launcher]: Starting browser Chrome
25 09 2017 13:56:19.263:INFO [HeadlessChrome 0.0.0 (Linux 0.0.0)]: Connected on socket zDM7md2A7Je8kqsPAAAA with id 1679961
25 09 2017 13:58:19.552:WARN [HeadlessChrome 0.0.0 (Linux 0.0.0)]: Disconnected (1 times), because no message in 120000 ms.
HeadlessChrome 0.0.0 (Linux 0.0.0) ERROR
  Disconnected, because no message in 120000 ms.

Here is my configuration:

const ChromiumRevision = require('puppeteer/package.json').puppeteer.chromium_revision;
const Downloader = require('puppeteer/utils/ChromiumDownloader');
const revisionInfo = Downloader.revisionInfo(Downloader.currentPlatform(), ChromiumRevision);

process.env.CHROME_BIN = revisionInfo.executablePath;
module.exports = function (config) {
    config.set({
        basePath: 'build/',
        frameworks: ['jasmine'],
        plugins: [
            require('karma-jasmine'),
            require('karma-chrome-launcher'),
        ],
        files: ['tests.js'],
        customLaunchers: {
            'ChromeHeadless': {
                base: 'Chrome',
                flags: [
                    '--headless',
                    '--disable-gpu',
                    // Without a remote debugging port, Google Chrome exits immediately.
                    '--remote-debugging-port=9222'
                ],
                debug: true
            }
        },
        autoWatch: false,
        browsers: ['ChromeHeadless'],
        singleRun: true,
        browserNoActivityTimeout: 120000,
        urlRoot: '/build/'
    })
}

Using karma-chrome-launcher: "2.1.1". Tried with the latest 2.2.0 version too. Tried with karma: "1.5.0", "1.6.0", and the latest "1.7.1".

I tried other flags like "--no-sandbox", "--disable-web-security" as suggested on a bunch of issues on karma repo. But still no luck. Is there a fix or workaround for this issue? Is there a reason why it's not working on the Linux server [Ubuntu Server 14.04.5 LTS].

[vargarobert]

Can someone address this please. I've tried so many karma configurations, using different package versions etc to make it work, but still no luck.

[ghost]

The way that you define CHROME_BIN has been updated in recent version (see the readme for more details). It is now:

process.env.CHROME_BIN = require('puppeteer').executablePath();

[vargarobert]

@saimaheshgaya this basically reaches the same result

[ghost]

try npm install && npm rebuild && npm test This worked for me

Also adding --no-sandbox to the flag list helps

[applecool]

@saimaheshgaya That is not going to resolve the issue. If you increase the timeout to x, it times out after those x ms.

[andrewl-telnyx]

npm install && npm rebuild && npm test

const process = require('process');
process.env.CHROME_BIN = require('puppeteer').executablePath();

//later:
 browsers: ['HeadlessChrome'],
    customLaunchers:{
      HeadlessChrome:{
        base: 'ChromeHeadless',
        flags: ['--no-sandbox']
      }
    },

appears to have worked for us.

[ghost]

npm rebuild is the key if you are switching platform. It recompiles the C++ addons for Node JS.

[vargarobert]

Still no luck here.

[applecool]

Same here! Hey @vargarobert I have posted the issue on the puppeteer's repo and they closed mine asking me to remove karma and try it out. I have to do that. Link here. Maybe try that out.

[vargarobert]

@applecool Pupetteer works as expected, just tried it out. It must be something related to karma-chrome-launcher not launching the headless browser.

[applecool]

@vargarobert That's cool. So, its clearly a problem with the karma-chrome-launcher then. I hope this problem gets the attention to the dev's on the team.

[kumvem]

@applecool The launcher is starting incorrect browser "Starting browser Chrome". it should launch "ChromeHeadless" Check my log below (mine is OSX as well, not Linux). When logs start flushing from HeadlessChrome 0.0.0 Google chromeheadless stated its execution, means Karma-chrome-launcher is fine. Did you report this to google chrome headless?

@applecool @vargarobert Puppeteer is not mandatory for ChromeHeadless browser in Karma. I can successfully run my tests without puppeteer.

package.json "karma": "^1.7.1", "karma-chrome-launcher": "^2.2.0",

my karma.conf.js starts with module.exports = function (config) { config.set({

My browser in karma.conf.js browsers: ['ChromeHeadless']

Im not using any custom launcher configurations. All options you have given are handled by karma-chrome-launcher line 168 in "node_modules/karma-chrome-launcher/index.js"

This is my log. [exec] Running "karma:unit" (karma) task [exec] 09 10 2017 22:52:13.282:INFO [karma]: Karma v1.7.1 server started at http://0.0.0.0:8090/ [exec] 09 10 2017 22:52:13.283:INFO [launcher]: Launching browser ChromeHeadless with unlimited concurrency [exec] 09 10 2017 22:52:13.289:INFO [launcher]: Custom Starting browser ChromeHeadless [exec] 09 10 2017 22:52:13.639:INFO [HeadlessChrome 0.0.0 (Mac OS X 10.12.6)]: Connected on socket D6nT8-N4aXCaoUpKAAAA with id 86242225

[applecool]

Thanks a lot @kumvem for the information. Yes, I did report it on the puppeteer.

I didn't had any problems on OSX either. And the log which I shared is from the linux execution not OSX. Thanks for pointing this out: Starting browser Chrome. I am not sure why that's getting showed up. I just tried to run the tests on OSX and in the logs, after ChromeHeadless is launched, It says the same Starting browser Chrome. Here's the latest log on OSX:

11 10 2017 09:44:41.589:INFO [karma]: Karma v1.6.0 server started at http://0.0.0.0:9876/build/
11 10 2017 09:44:41.590:INFO [launcher]: Launching browser ChromeHeadless with unlimited concurrency
11 10 2017 09:44:41.595:INFO [launcher]: Starting browser Chrome
11 10 2017 09:44:41.919:INFO [HeadlessChrome 0.0.0 (Mac OS X 10.12.6)]: Connected on socket hyN1lr9EWJpmp63QAAAA with id 5427296
HeadlessChrome 0.0.0 (Mac OS X 10.12.6): Executed 1 of 202 SUCCESS (0 secs / 0.004 secs)

I am completely confused with that log. Nevertheless, all the tests execute successfully.

You have mentioned that you aren't using puppeteer and still be able to execute the tests with the chrome headless. So, I am assuming you installed Chrome GUI on your machine which is being launched by the karma-chrome-launcher with the headless flag (which presumably should be mentioned in the customLaunchers property). Is that right? And, in your provided config, I don't see the customLaunchers property. Could you please share that too.

I will try to run the tests with ChromeHeadless without the puppeteer and see what's going on.

Thank you again.

[applecool]

@kumvem I removed puppeteer, and also the customLaunchers property in the config. As the base property in the customLaunchers was assigned to Chrome, the Starting browser Chrome was logged. This is my latest config and the log:

module.exports = function (config) {
    config.set({
        basePath: 'build/',
        frameworks: ['jasmine'],
        plugins: [
            require('karma-jasmine'),
            require('karma-chrome-launcher'),
        ],
        files: ['tests.js'],
        autoWatch: false,
        browsers: ['ChromeHeadless'],
        singleRun: true,
        browserNoActivityTimeout: 120000,
        urlRoot: '/build/'
    })
}

Log on OSX:

11 10 2017 10:03:03.895:INFO [karma]: Karma v1.7.1 server started at http://0.0.0.0:9876/build/
11 10 2017 10:03:03.896:INFO [launcher]: Launching browser ChromeHeadless with unlimited concurrency
11 10 2017 10:03:03.900:INFO [launcher]: Starting browser ChromeHeadless
11 10 2017 10:03:04.165:INFO [HeadlessChrome 0.0.0 (Mac OS X 10.12.6)]: Connected on socket PiVHGIQOtKEzQkEFAAAA with id 85884203
HeadlessChrome 0.0.0 (Mac OS X 10.12.6): Executed 1 of 202 SUCCESS (0 secs / 0.003 secs)

Successfully runs on OSX and executes the tests without Puppeteer. No, flags, nothing. Works out of the box with just the browser set to ChromeHeadless.

Same config, Log when running on Linux Server:

11 10 2017 10:10:37.190:INFO [karma]: Karma v1.7.1 server started at http://0.0.0.0:9876/build/
11 10 2017 10:10:37.191:INFO [launcher]: Launching browser ChromeHeadless with unlimited concurrency
11 10 2017 10:10:37.196:INFO [launcher]: Starting browser ChromeHeadless
11 10 2017 10:10:37.946:INFO [HeadlessChrome 0.0.0 (Linux 0.0.0)]: Connected on socket 8VVqH7pkzV8Pk3kEAAAA with id 14558080
11 10 2017 10:12:38.264:WARN [HeadlessChrome 0.0.0 (Linux 0.0.0)]: Disconnected (1 times), because no message in 120000 ms.
  HeadlessChrome 0.0.0 (Linux 0.0.0) ERROR
  Disconnected, because no message in 120000 ms.
[10:12:38] 'test' errored after 2.03 min
[10:12:38] Error: Karma tests failed

😞

[applecool]

@vargarobert any luck?

[vargarobert]

I've tried numerous combinations on different platforms. It works fine on my mac, as it does with yours because you have Chrome installed. However, not on linux (teamcity CI for example). On Mac you can also notice the icon showing up in your dock for a few seconds even though the window doesn't actually show up.

I would like to be able to run it independently of the Chrome GUI installed (just like phantomJS, slient). You download a binary for your platform and run heedlessly. Simple - to work in any environment.

Here is where the problems start to appear. You set CHROME_BIN or CHROMIUM_BIN to your local chromium binary or puppeteer chromium binary and it doesn't lunch (not even when you use ChromiumHeadless, regardless of the platform and browser configuration - I've tried all of them).

We should try to get an actual chrome binary from somewhere and try it out. Like I said so far I've used puppeteer and local binary (downloaded from https://www.chromium.org/getting-involved/download-chromium)

[applecool]

Thanks for sharing the info @vargarobert I wish it was as simple as Phantom JS though.

[kumvem]

@applecool I got timeout issue in Mac as well. I added 'captureTimeout' in karma.conf.js to solve the issue. On my Linux box upgraded NPM and Node to latest version to solve the issue as puppeteer need node 6+

[applecool]

@kumvem I didn't get the timeout issue on Mac at all. seems like you misunderstood. I am getting timeout problem when executing on linux box.

[michaelkrone]

@applecool We got the same problem: locally the tests run, on the Linux CI, chrome timed out randomly. Increasing the browserNoActivityTimeout in the karma.conf to a very high value (in our case 60000) did the trick.

[applecool]

@michaelkrone I already have 120000ms. I re-tried it with much higher value of 3 and 5 minutes too. It's still the same. It just times out. And I have also tried on the Linux GUI (Ubuntu Desktop 16.0.4 LTS). Still the same. Doesn't work with ChromeHeadless. It's so annoying 😞

[reduckted]

The workaround posted by @andrewl-telnyx appears to be working for me. I'm not using puppeteer. All I had to do was add this to my config:

browsers: ['HeadlessChrome'],
customLaunchers:{
    HeadlessChrome:{
        base: 'ChromeHeadless',
        flags: ['--no-sandbox']
    }
}

It always timed out when using ChromeHeadless as the browser, and always succeeds when using the custom HeadlessChrome. Now I just have to work out what the ramifications are for using --no-sandbox.

Edit: I may have spoken too soon. It's also timing out, but does occasionally succeed. 😢

[applecool]

@reduckted Which OS are you on? Linux or OSX?

[reduckted]

Sorry, should have mentioned that. I'm actually on Windows 10.

[neutraali]

@applecool FWIW we have a working Linux Mint (Ubuntu) and Alpine config using puppeteer, which I'll post here if it might help at all. This assumes that you have CHROME_BIN set with puppeteer:

process.env.CHROME_BIN = puppeteer.executablePath();

... And then as to the actual Karma config:

var KarmaConfig = {
. . .
browsers: ['ChromeHeadlessCustom'],
plugins: 'karma-chrome-launcher',
customLaunchers: {
    'ChromeHeadlessCustom': {
        base: 'ChromiumHeadless'
        flags: [
            '--no-sandbox',
            '--headless',
            '--disable-gpu',
            '--disable-translate',
            '--disable-extensions'
        ]
    }
}

This is done using:

"karma": "1.7.1",
"karma-chrome-launcher": "2.2.0",
"puppeteer": "0.13.0"

... Our problem is the reverse. We can't get ChromeHeadlessCustom to work on OSX. Same timeout problem ... :)

UPDATE: We also got it working on mac by switching the base to ChromeHeadless instead of ChromiumHeadless (when running the tests on OSX).

[corsen2000]

Having the same issue on Windows 7 ChromeHeadless have not captured in 60000 ms, killing..

I've tried all of the flags listed in this issue, but non help it connect. Task manager shows that Chromium is running, but for some reason it is not connecting to karma.

Non headless chrome works fine.

[bas-l]

UPDATE: My issue was solved using the answer mentioned here https://github.com/karma-runner/karma/issues/2652#issuecomment-335599121

I have the same issue on Windows 7. Tried with all flags and also with the custom launcher. No luck.

    [36m08 01 2018 12:21:08.480:DEBUG [config]: [39mautoWatch set to false, because of singleRun
    [36m08 01 2018 12:21:08.483:DEBUG [plugin]: [39mLoading inlined plugin (defining framework:jasmine).
    [36m08 01 2018 12:21:08.483:DEBUG [plugin]: [39mLoading inlined plugin (defining launcher:Chrome, launcher:ChromeHeadless, launcher:ChromeCanary, launcher:ChromeCanaryHeadless, launcher:Chromium, launcher:ChromiumHeadless, launcher:Dartium, test).
    [36m08 01 2018 12:21:08.483:DEBUG [plugin]: [39mLoading inlined plugin (defining reporter:junit).
    [36m08 01 2018 12:21:08.483:DEBUG [plugin]: [39mLoading inlined plugin (defining reporter:coverage-istanbul).
    [36m08 01 2018 12:21:08.483:DEBUG [plugin]: [39mLoading inlined plugin (defining framework:@angular/cli, preprocessor:@angular/cli, reporter:@angular/cli, middleware:angularCliBlocker).
    [36m08 01 2018 12:21:08.483:DEBUG [plugin]: [39mLoading inlined plugin (defining launcher:MyHeadlessChrome).
    [36m08 01 2018 12:21:08.640:DEBUG [web-server]: [39mInstantiating middleware
    [36m08 01 2018 12:21:08.641:DEBUG [reporter]: [39mTrying to load reporter: @angular/cli
    [36m08 01 2018 12:21:08.642:DEBUG [reporter]: [39mTrying to load color-version of reporter: @angular/cli (@angular/cli_color)
    [36m08 01 2018 12:21:08.642:DEBUG [reporter]: [39mCouldn't load color-version.
    [36m08 01 2018 12:21:08.653:DEBUG [reporter]: [39mTrying to load reporter: junit
    [36m08 01 2018 12:21:08.654:DEBUG [reporter]: [39mTrying to load color-version of reporter: junit (junit_color)
    [36m08 01 2018 12:21:08.654:DEBUG [reporter]: [39mCouldn't load color-version.
    [32m08 01 2018 12:21:13.741:INFO [karma]: [39mKarma v1.7.1 server started at http://0.0.0.0:9876/
    [32m08 01 2018 12:21:13.742:INFO [launcher]: [39mLaunching browser MyHeadlessChrome with unlimited concurrency
    [32m08 01 2018 12:21:13.748:INFO [launcher]: [39mStarting browser Chrome
    [36m08 01 2018 12:21:13.748:DEBUG [temp-dir]: [39mCreating temp dir at C:\Users\BAS~1.LUK\AppData\Local\Temp\karma-40242111
    [36m08 01 2018 12:21:13.749:DEBUG [launcher]: [39mC:\Program Files (x86)\Google\Chrome\Application\chrome --user-data-dir=C:\Users\BAS~1.LUK\AppData\Local\Temp\karma-40242111 --no-default-browser-check --no-first-run --disable-default-apps --disable-popup-blocking --disable-translate --disable-background-timer-throttling --disable-renderer-backgrounding --disable-device-discovery-notifications --headless --disable-gpu  --remote-debugging-port=9222 --enable-logging --v=1 --no-sandbox --disable-web-security --disable-setuid-sandbox http://localhost:9876/?id=40242111
    [36m08 01 2018 12:21:23.995:DEBUG [web-server]: [39mserving: C:\repo\nano\app\node_modules\karma\static/client.html
    [36m08 01 2018 12:21:28.502:DEBUG [web-server]: [39mserving: C:\repo\nano\app\node_modules\karma\static/karma.js
    [36m08 01 2018 12:21:30.781:DEBUG [karma]: [39mA browser has connected on socket J4snil3YHSxA4W_rAAAA
    [36m08 01 2018 12:21:33.465:DEBUG [karma]: [39mA browser has connected on socket Nnd2FegD0gKJcjhbAAAB
    [36m08 01 2018 12:21:40.893:DEBUG [karma]: [39mA browser has connected on socket 64QidqbZxK5TQnmCAAAC
    [36m08 01 2018 12:21:42.832:DEBUG [karma]: [39mA browser has connected on socket NZiimJP4jpkHRFMMAAAD
    [36m08 01 2018 12:21:48.757:DEBUG [karma]: [39mA browser has connected on socket GQL7_8T7X3cong11AAAE
    [36m08 01 2018 12:22:01.865:DEBUG [karma]: [39mA browser has connected on socket vBzAg2xKLlqHml40AAAF
    [36m08 01 2018 12:22:08.319:DEBUG [karma]: [39mA browser has connected on socket qJqcGZVouDEe0Ci7AAAG
    [33m08 01 2018 12:22:13.756:WARN [launcher]: [39mChrome have not captured in 60000 ms, killing.
    [36m08 01 2018 12:22:13.765:DEBUG [launcher]: [39mProcess Chrome exited with code 0
    [36m08 01 2018 12:22:13.766:DEBUG [temp-dir]: [39mCleaning temp dir C:\Users\BAS~1.LUK\AppData\Local\Temp\karma-40242111

[bennypowers]

Posting for posterity. I had this same issue with a project dependent on Karma 1.7.0, so I switched from ChromeHeadless to Chrome and noticed that the test runner launched Chrome to another local project running it's own local webserver running on the same port Karma expected (8080). After killing the unrelated local dev server that was running on 8080, and switching back to ChromeHeadless, everything was fine.

tl;dr: make sure you kill any servers running locally on your karma server's port (usually 8080)

[eppsilon]

Recently Karma became much less reliable for my team's project. I recently rebuilt the Docker image for our build agents, and I think it picked up a newer version of Chrome along the way. I noticed that karma-chrome-launcher passes --disable-gpu even though that is only recommended for running headless Chrome on Windows. So I tried adding --enable-gpu to counteract that, and so far it appears to be working.

Current Chrome version: 63.0.3239.132

Custom launcher:

ChromeHeadlessNoSandbox: {
  base: 'ChromeHeadless',
  flags: [
    '--no-sandbox', // required to run without privileges in Docker
    '--disable-web-security',
    '--enable-gpu'
  ]
}

[applecool]

To all, when you run the tests on a linux machine, what version number do you see? I always see this: HeadlessChrome 0.0.0 (Linux 0.0.0) Does anyone see the same 0.0.0 version number?

Update on the tests on Headless: I tried all the above suggestions in my project and sadly I am still unable to run the tests on a linux server. I tried tweaking the main.js from karma node modules as suggested here on this thread which worked for a bunch of people. You can take a look at it, maybe it might be helpful for you who are still searching for a solution. For some reason, I strongly believe that the issue is caused because of the SSL certificates on my end (esp. on linux) which is not yet supported by Chrome Headless yet. acceptInsecureCerts is the flag which they are going to introduce. It will be out soon as a part of Chrome 66 release according to the issue tracker here. As of now, I couldn't test that out because Chrome Canary builds are not available for Linux. I did try running the tests on the Chrome latest unstable dev build which is 65.0.3322.3 which didn't give me any fruitful results.

If anyone has any other suggestions for fixes, please do comment here. It is very valuable if it helps at least one of us who is looking for a solution :)