Error ERR_STREAM_WRITE_AFTER_END when running in Kubernetes AWS EKS

[coveraver]

Describe the bug I'm trying to set up interaction with browserless in a k8s cluster. Browserless is installed in one pod, the script for the test is installed in the other. I could not get a working version with python and playwright, so I decided to use nodejs for testing. Script Example

const puppeteer = require('puppeteer');


const browserInit = async () => await puppeteer.connect({
    browserWSEndpoint: 'ws://headless-browserless:3000?timeout=120000',
    defaultViewport: {
        width: 1280,
        height: 800
    }})

async function start() {
    const browser = await browserInit();
    const page = await browser.newPage();

    await page.goto('https://google.com', { waitUntil: 'networkidle0' });
    await page.screenshot({ path: 'example.png' });
    return browser.close();
}

start();

The script connects but gets an error after timeout

Error [ERR_STREAM_WRITE_AFTER_END]: write after end
    at new NodeError (node:internal/errors:393:5)
    at _write (node:internal/streams/writable:322:11)
    at Writable.write (node:internal/streams/writable:337:10)
    at Socket.ondata (node:internal/streams/readable:766:22)
    at Socket.emit (node:events:513:28)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Readable.push (node:internal/streams/readable:234:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
2023-05-31T12:52:13.733Z browserless:chrome-helper Temp dir /tmp/browserless-data-dir-7A2R9d removed successfully
2023-05-31T12:52:13.781Z browserless:chrome-helper Garbage collecting and removing listeners

Deployment and service example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: debug-chrome
  namespace: default
  labels:
    app: debug-chrome
spec:
  selector:
    matchLabels:
      app: debug-chrome
  replicas: 1
  strategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: debug-chrome
    spec:
      containers:
      - name: debug-chrome
        image: browserless/chrome:1-puppeteer-1.20.0
        ports:
          - containerPort: 3000
---
apiVersion: v1
kind: Service
metadata:
  name: debug-chrome
  namespace: default
spec:
  selector:
    app: debug-chrome
  ports:
    - port: 3000
      targetPort: 3000

To Reproduce Steps to reproduce the behavior:

1. Deploy browserless into k8s
2. Deploy nodejs image
3. Create service for browserless pod
4. Run script from nodejs pod

Expected behavior Successful screenshot capture

[coveraver]

Same error with sidecar container

[omri-shilton]

im also having the same issue. in my case im trying to reach the pod from a lambda thats going through a NLB ingress.

[byt3bl33d3r]

Same issue here with Docker compose

[byt3bl33d3r]

After some debugging, this error seems to stem from the CONNECTION_TIMEOUT parameter (https://www.browserless.io/docs/docker#connection-timeout). Setting CONNECTION_TIMEOUT to -1 in my docker compose file solved this for me (-1 turns the timeout off completely, adjust accordingly).

The CONNECTION_TIMEOUT parameter is set to 30s by default meaning if you navigate to a page (and don't set the default page navigation timeout to less than 30s) Browserless will just disconnect your entire session which results in the ERR_STREAM_WRITE_AFTER_END error. You have to either turn the connection timeout off completely or make sure that your default navigation timeout is less than 30s.

[joelgriffith]

Yes -- this sometimes comes from our server forcing the connection closed once a timeout is reached. Typically it's ok but somewhat of an annoyance.