Support launching in Docker

[nathanboktae]

We have a CI environment at my company that does not have chrome nor allows apt-get install on our machines, but docker is available for better isolation of these dependencies, but I see no documentation or support for Docker. This would be greatly appreciated. I'm trying to get a shell script put together that can achieve this by passing in as a CHROME_PATH, which may be the best, most unobtrustive way to support it. I'm struggling to get it working though :/

[patrickhulce]

thanks for the feedback @nathanboktae, these docs used to be more closely tied to the ones in Lighthouse that reference getting setup in CI/docker environments

this dockerfile was once linked in the docs there for getting everything setup, but we'll have to figure out the dedicated chrome-launcher documentation story here :)

[nathanboktae]

The tricky thing seems to be intercepting --remote-debugging-port and --user-data-dir to also map those ports and folders over in the docker run command.

[jakelacey2012]

@patrickhulce don't know if you can help me here but I'm having trouble in getting this to work in docker also. I'm trying to run a script which launches chrome then drives the browser using puppeteer.

  const chrome = await chromeLauncher.launch(opts);
  opts.port = chrome.port;
  
  // Connect to it using puppeteer.connect().
  const resp = await util.promisify(request)(`http://localhost:${opts.port}/json/version`); 
  const {webSocketDebuggerUrl} = JSON.parse(resp.body);
  const browser = await puppeteer.connect({browserWSEndpoint: webSocketDebuggerUrl});

However I seem to be getting this error.

error { e: { TypeError [ERR_INVALID_URL]: Invalid URL: undefined
    at onParseError (internal/url.js:219:17)
    at parse (internal/url.js:228:3)
    at new URL (internal/url.js:311:5)
    at WebSocket.initAsClient (/app/node_modules/puppeteer/node_modules/ws/lib/websocket.js:470:27)
    at new WebSocket (/app/node_modules/puppeteer/node_modules/ws/lib/websocket.js:62:20)
    at Promise (/app/node_modules/puppeteer/lib/WebSocketTransport.js:28:18)
    at new Promise (<anonymous>)
    at Function.create (/app/node_modules/puppeteer/lib/WebSocketTransport.js:27:12)
    at Launcher.connect (/app/node_modules/puppeteer/lib/Launcher.js:285:44)
    at module.exports.connect (/app/node_modules/puppeteer/lib/Puppeteer.js:44:27) input: 'undefined' } }
^CTue, 29 Jan 2019 17:33:14 GMT ChromeLauncher Killing Chrome instance 63

My dockerfile looks like this,

FROM node:8-slim

RUN apt-get update && \
apt-get install -yq gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 \
libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 \
libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 \
libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 \
ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget chromium  && \
wget https://github.com/Yelp/dumb-init/releases/download/v1.2.1/dumb-init_1.2.1_amd64.deb && \
dpkg -i dumb-init_*.deb && rm -f dumb-init_*.deb && \
apt-get clean && apt-get autoremove -y && rm -rf /var/lib/apt/lists/*

RUN yarn global add [email protected] && yarn cache clean

ENV NODE_PATH="/usr/local/share/.config/yarn/global/node_modules:${NODE_PATH}"

ENV PATH="/tools:${PATH}"

# ADD ./tools /tools
# RUN chmod +x /tools/* && mkdir /screenshots

WORKDIR /app

COPY package.json /app
COPY yarn.lock /app

RUN yarn

COPY /src /app/src

ENTRYPOINT ["dumb-init", "--"]
CMD ["yarn", "start"]

Do you know where I could be going wrong or where I could look next?

[patrickhulce]

@jakelacey2012 what does resp.body actually look like in that script?

You're passing undefined to puppeteer as the websocket URL so something is amiss.

[jakelacey2012]

Thanks for getting back to me.

resp.body looks like this

{ Browser: '',
  'Protocol-Version': '1.2',
  'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome Safari/537.36',
  'V8-Version': '5.7.492.63',
  'WebKit-Version': '537.36 (@a6a06b78087c9fdb4b12fe0ac1b87fdc10179f8b)' }

from what I can see from printing the chrome object is that it has a PID

{ chrome:
   { pid: 63,
     port: 40283,
     kill: [Function: kill],
     process:
      ChildProcess {
        domain: null,
        _events: {},
        _eventsCount: 0,
        _maxListeners: undefined,
        _closesNeeded: 1,
        _closesGot: 0,
        connected: false,
        signalCode: null,
        exitCode: null,
        killed: false,
        spawnfile: '/usr/bin/chromium',
        _handle: [Object],
        spawnargs: [Array],
        pid: 63,
        stdin: null,
        stdout: null,
        stderr: null,
        stdio: [Array] } } }

[jakelacey2012]

yeah its a little weird because its not like the service at that URL isn't running from what I can see because if I remove /version from the end of it resp.body now contains this.

[ { description: '',
    devtoolsFrontendUrl: '/devtools/inspector.html?ws=localhost:41349/devtools/page/ebb8bd00-3a20-41ca-9c38-1fea53f05c30',
    id: 'ebb8bd00-3a20-41ca-9c38-1fea53f05c30',
    title: 'about:blank',
    type: 'page',
    url: 'about:blank',
    webSocketDebuggerUrl: 'ws://localhost:41349/devtools/page/ebb8bd00-3a20-41ca-9c38-1fea53f05c30' } ]

and I've tried selecting webSocketDebuggerUrl but I get another error when I do this, so I don't think that is the answer.

error { e:
   { Error: Protocol error (Target.getBrowserContexts): 'Target.getBrowserContexts' wasn't found
    at Promise (/app/node_modules/puppeteer/lib/Connection.js:73:56)
    at new Promise (<anonymous>)
    at Connection.send (/app/node_modules/puppeteer/lib/Connection.js:72:12)
    at Launcher.connect (/app/node_modules/puppeteer/lib/Launcher.js:289:50)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:189:7)

specifically.

[patrickhulce]

@jakelacey2012 that means you're using a version of chrome that's not compatible with the version of puppeteer you're using. Each version of puppeteer is meant to be used with a very specific, fixed Chrome version.

[jakelacey2012]

Ahh interesting!

Do you know how I can find the right chrome version?

In my dockerfile I'm specifying puppeteer 1.0.0 but my package.json i'm using ^1.1.0.

    "chrome-launcher": "^0.10.5",
    "lighthouse": "^4.1.0",
    "lodash": "^4.17.11",
    "puppeteer": "^1.11.0",
    "request": "^2.88.0"

(The reason why I have different versions is because I had previous issue which one of my last projects and I had to use a specific version of puppeteer)

[jakelacey2012]

Ahh https://github.com/GoogleChrome/puppeteer/blob/master/package.json#L11 hmm not sure how I can check or install this version though.

[patrickhulce]

Puppeteer automatically downloads a compatible version and sticks it in your node_modules for you, this is what it will use by default. Either way I suggest you find help over in their repo. You don't really need to use this library at all if you're primarily interested in using puppeteer :)

[jakelacey2012]

Thanks for the help, I got it working. Really appreciate the help.

Yeah its not immediately obvious why I'm using chrome-launcher from my explanation above, but the reason is because I want to leverage lighthouse to easily get metrics and from the examples I've seen is you need to link them in someway, I don't completely understand the why yet.

And for any future travlers https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md#running-puppeteer-in-docker this helped.

[kyasbal]

I struggled with the relevant issue for a while.

This is my code to launch Chrome in headless mode in Docker.

function getChromeFlags(resolution: Resolution2D) {
  let appendedOptions = headless ? ["--headless", "--disable-gpu"] : [];
  return [
    `--window-size=${resolution.width},${resolution.height}`,
    ...appendedOptions
  ];
}
const chrome = await ChromeLauncher.launch({
  startingUrl: url,
  chromeFlags: getChromeFlags(resolution)
});

Then I got this error.

{ Error: connect ECONNREFUSED 127.0.0.1:46803
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1107:14)
  errno: 'ECONNREFUSED',
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 46803 }

I tried to use puppeteer workaround that @jakelacey2012 suggested. But actually that won't work for me.

I could solve this problem by adding --no-sandbox flag as chromeFlags in the launch option.

I hope this workaround is still safe for most of CI use case.

[patrickhulce]

@kyasbal-1994 the lighthouse-ci docs have a decent explanation of the tradeoffs involved in --no-sandbox and the different workarounds.