ECONNREFUSED ::1:4400 error even though firestore port and host are explicitly specified

[Acterion]

[REQUIRED] Environment info

firebase-tools: 11.2.2

Platform: macOS

[REQUIRED] Test case

//firestore.spec.ts
import * as firebaseTesting from "@firebase/rules-unit-testing";
import { before } from "mocha";
before(async () => {
  env = await firebaseTesting.initializeTestEnvironment({
    projectId: "demo-project",
    firestore: {
      host: "127.0.0.1",
      port: 8080,
      rules: fs.readFileSync("firestore.rules", "utf8"),
    },
  });
});

//package.json
{
  "name": "demo-project/security-rules",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "scripts": {
    "test:mocha": "mocha test/*.spec.ts -r=ts-node/register --timeout 10000",
    "test": "firebase emulators:exec --only firestore \"yarn test:mocha\"",
  },
  "dependencies": {
    "firebase-tools": "11.2.2"
  },
  "devDependencies": {
    "@firebase/rules-unit-testing": "2.0.3",
    "@types/mocha": "9.1.1",
    "firebase": "9.9.0",
    "firebase-admin": "10.3.0",
    "mocha": "10.0.0",
    "ts-node": "10.8.2",
    "typescript": "4.7.4"
  }
}

[REQUIRED] Steps to reproduce

Run yarn run test

[REQUIRED] Expected behavior

Expected to exit with code 0

[REQUIRED] Actual behavior

Fails with FetchError: request to http://localhost:4400/emulators failed, reason: connect ECONNREFUSED ::1:4400

This error was introduced in release 11.0.0 and is not present in 10.9.2 Seems like it still tries to find emulators hub, even though host and port for firestore are explicitly specified. Same error can be observed if a flag FIRESTORE_EMULATOR_HOST=127.0.0.1:8080 is passed.

Currently I solved this error in my project by locking firebase-tools to 10.9.2

[bkendall]

Alright, so this is known, but hard to address fully.

In short, it has to do with how DNS on a machine is being resolved in Node. Most of the emulators bind to the ipv4 addresses, but if a process resolves localhost to the ipv6 address [::1] that can cause the connections to fail (since emulators don't tend to bind on the ipv6 address).

@yuchenshi has been looking into addressing this in a couple places around the emulators. Working on addressing this, but it's been difficult.

[augustmuir]

Same issue here on latest version, downgrade to 10.9.2 solved the problem. This is the error message of ui-debug.log when I tried running the latest version:

u [FetchError]: request to http://localhost:4400/emulators failed, reason: connect ECONNREFUSED ::1:4400
    at ClientRequest.<anonymous> (C:\Users\august\.cache\firebase\emulators\ui-v1.7.0\server.bundle.js:326:16909)
    at ClientRequest.emit (node:events:513:28)
    at Socket.socketErrorListener (node:_http_client:481:9)
    at Socket.emit (node:events:513:28)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  type: 'system',
  errno: 'ECONNREFUSED',
  code: 'ECONNREFUSED'
}

I feel like if there was a option to change the Hub port it might resolve this as I've seen the same error before when using a "blocked" port, and it was solved by using another.

[valle-xyz]

+1

What is the preferred method to downgrade firebase globally?

[yuchenshi]

~Fixed in v11.3.0.~

[yuchenshi]

Whoops, wrong issue -- sorry for the confusion. We've fixed some other issues related to Emulator UI, not Emulator hub (used in testing).

[yuchenshi]

While we work on this, I think a better workaround than downgrading Firebase CLI is specifying the hub address explicitly using IPv4:

e.g. change the command:

firebase emulators:exec --only firestore \"FIREBASE_EMULATOR_HUB=127.0.0.1:4400 yarn test:mocha\"

Or in configuration:

  env = await firebaseTesting.initializeTestEnvironment({
    projectId: "demo-project",
    /* firestore and other emulators omitted, new hub options below */
    hub: {
      host: "127.0.0.1",
      port: 4400,
    },
  });

[valle-xyz]

👏🦾❤️

Upgrading to 11.3.0 fixed this for me! Thanks, @yuchenshi !

[kris-kolve-jumio]

setting the Hub host resolved the ECONNREFUSED error for me however, now every test I run storage tests... all the test time out using Mocha at 20seconds.

If I set the storage configuration with host: "127.0.0.1", port:9199, I can get tests with getDownloadURL() to complete appropriately. However, tests with uploadBytes still timeout.

In fact I am not able to get any upload statements to work with the emulators!

[yuchenshi]

@kris-kolve-jumio Would you mind opening a different issue for the storage emulator timeout?

[vdiaz1130]

@kris-kolve-jumio Would you mind opening a different issue for the storage emulator timeout?

https://github.com/firebase/firebase-tools/issues/4908

[pph7]

+1

[yuchenshi]

Hi all, we've updated how the emulator listens on ports in v11.14.2 and we believe it now should work for most cases (including the OP's repro, quickstart-testing, emulators-codelab, etc.) without any need of workarounds. Please let us know if this isn't the case for you even after the upgrade.

To provide more details, we've changed the Emulator Hub to listen on both IPv4 and IPv6 address by default (if possible) (#5088), and the discovery now returns 127.0.0.1 (IPv4) instead of the ambiguous localhost (which may resolve to IPv6 on Node.js 18+) for other emulators (#5083). These changes together should prevent ECONNREFUSED ::1:4400 and remove the need to explicitly specify host for anything other than the hub. Feel free to dig into the PRs if you're interested in the technical details.

[modosc]

hi @yuchenshi - v11.14.2 doesn't work for us in docker.

with this config:

{
  "firestore": {
    "rules": "firestore.rules"
  },
  "emulators": {
    "firestore": {
      "port": 8080,
      "host": "0.0.0.0"
    },
    "pubsub": {
      "port": 8085,
      "host": "0.0.0.0"
    },
    "ui": {
      "enabled": true,
      "port": 4000,
      "host": "0.0.0.0"
    }
  }
}

we get:

[debug] [2022-10-11T21:45:42.494Z] Error: listen EADDRNOTAVAIL: address not available ::1:4400
    at Server.setupListenHandle [as _listen2] (node:net:1415:21)
    at listenInCluster (node:net:1480:12)
    at doListen (node:net:1629:7)
    at processTicksAndRejections (node:internal/process/task_queues:84:21)

and adding this block:

    "hub": {
      "host": "0.0.0.0",
      "port": 4400
    }

gives us a new failure:

[debug] [2022-10-11T21:53:45.578Z] Error: listen EADDRNOTAVAIL: address not available ::1:4500
    at Server.setupListenHandle [as _listen2] (node:net:1415:21)
    at listenInCluster (node:net:1480:12)
    at doListen (node:net:1629:7)
    at processTicksAndRejections (node:internal/process/task_queues:84:21)

[yuchenshi]

@modosc Thanks for the follow-up. Would it be possible to get the full firebase-debug.log? We'd need to look into where this happens in context and see if we can find a workaround.

[yuchenshi]

@modosc I have a guess on what's going on here -- the error may be happening with our new port detection logic and somehow IPv6 isn't available in your Docker network. But we'd still appreciate full logs to confirm this.

Also, if that's the case, I think you can work around this by adding logging too (before we make the next release). Let us know if it works for you.

    "hub": {
      "host": "0.0.0.0",
      "port": 4400
    },
    "logging": {
      "host": "0.0.0.0",
      "port": 4500
    }

[fabfuel]

It is a IPv6 issue within the Docker container for us too @yuchenshi

If someone is emulating functions as well, you have to define eventarc's host as well to make it work again:

    "hub": {
      "host": "0.0.0.0",
      "port": 4400
    },
    "logging": {
      "host": "0.0.0.0",
      "port": 4500
    },
    "eventarc": {
      "host": "0.0.0.0",
      "port": 9299
    }

The full debug log:

[debug] [2022-10-12T09:44:05.420Z] ----------------------------------------------------------------------
[debug] [2022-10-12T09:44:05.422Z] Command:       /usr/local/bin/node /usr/local/bin/firebase emulators:start --project project-e2e --only firestore,storage,auth,functions
[debug] [2022-10-12T09:44:05.423Z] CLI Version:   11.14.2
[debug] [2022-10-12T09:44:05.423Z] Platform:      linux
[debug] [2022-10-12T09:44:05.423Z] Node Version:  v16.15.1
[debug] [2022-10-12T09:44:05.423Z] Time:          Wed Oct 12 2022 09:44:05 GMT+0000 (Coordinated Universal Time)
[debug] [2022-10-12T09:44:05.424Z] ----------------------------------------------------------------------
[debug]
[debug] [2022-10-12T09:44:05.540Z] > command requires scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","https://www.googleapis.com/auth/cloud-platform"]
[debug] Failed to authenticate, have you run firebase login?
[warn] ⚠  emulators: You are not currently authenticated so some features may not work correctly. Please run firebase login to authenticate the CLI.
[debug] [2022-10-12T09:44:05.679Z] openjdk version "11.0.16.1" 2022-08-12
[debug] [2022-10-12T09:44:05.679Z]
OpenJDK Runtime Environment (build 11.0.16.1+1-alpine-r0)
OpenJDK 64-Bit Server VM (build 11.0.16.1+1-alpine-r0, mixed mode)

[debug] [2022-10-12T09:44:05.683Z] Parsed Java major version: 11
[info] i  emulators: Starting emulators: auth, functions, firestore, storage {"metadata":{"emulator":{"name":"hub"},"message":"Starting emulators: auth, functions, firestore, storage"}}
[debug] [2022-10-12T09:44:05.688Z] assigned listening specs for emulators {"user":{"hub":[{"address":"0.0.0.0","family":"IPv4","port":4400}],"ui":[{"address":"0.0.0.0","family":"IPv4","port":4000}],"logging":[{"address":"0.0.0.0","family":"IPv4","port":4500}],"auth":[{"address":"0.0.0.0","family":"IPv4","port":9099}],"firestore":[{"address":"0.0.0.0","family":"IPv4","port":8080}],"firestore.websocket":[{"address":"0.0.0.0","family":"IPv4","port":9150}],"storage":[{"address":"0.0.0.0","family":"IPv4","port":9199}]},"metadata":{"message":"assigned listening specs for emulators"}}
[debug] [2022-10-12T09:44:05.692Z] [hub] writing locator at /tmp/hub-project-e2e.json
[info] i  emulators: Shutting down emulators. {"metadata":{"emulator":{"name":"hub"},"message":"Shutting down emulators."}}
[info] i  hub: Stopping emulator hub {"metadata":{"emulator":{"name":"hub"},"message":"Stopping emulator hub"}}
[debug] [2022-10-12T09:44:05.698Z] Error: listen EADDRNOTAVAIL: address not available ::1:9299
    at Server.setupListenHandle [as _listen2] (node:net:1355:21)
    at listenInCluster (node:net:1420:12)
    at doListen (node:net:1559:7)
    at processTicksAndRejections (node:internal/process/task_queues:84:21)
[error]
[error] Error: An unexpected error has occurred.

[modosc]

i can confirm that adding both of these blocks fixes the issue for us:

    "hub": {
      "host": "0.0.0.0",
      "port": 4400
    },
    "logging": {
      "host": "0.0.0.0",
      "port": 4500
    },

i can also confirm the exact same log output as above. running docker on mac and there's no ipv6 support.

[yuchenshi]

EADDRNOTAVAIL should be fixed in v11.14.3 (no workaround needed any more). Thanks for the reports.

[joshtemple]

@yuchenshi I'm still encountering this issue in v11.15.0 in Docker. Adding the logging section referenced above to my firebase.json does fix the problem.

[greg-md]

The same issue happens inside a github action. Are there any fixes for that? Specifying the host to 0.0.0.0 doesn't help. It still says connect ECONNREFUSED ::1:8080

PS: Downgrading to node 16 fixed the issue.

[gkawin]

The same issue happens inside a github action. Are there any fixes for that? Specifying the host to 0.0.0.0 doesn't help. It still says connect ECONNREFUSED ::1:8080

PS: Downgrading to node 16 fixed the issue.

TRY THIS. In the packages.json.I think Its from this issue. https://github.com/nodejs/node/issues/41145

So, I added node runtime options in my package.json file like this. ... "test": "node --experimental-vm-modules --dns-result-order=ipv4first node_modules/jest/bin/jest.js", "integration:test": "firebase emulators:exec --only firestore,functions --project {PROJECT_ID} "pnpm test"" ...

[dr-aiuta]

Same issue here, node 18.15.0

[christiangenco]

For me the problem was fixed by changing host: localhost to host: 127.0.0.1 in my initializeTestEnvironment function call.

From this:

testEnv = await initializeTestEnvironment({
  projectId: PROJECT_ID,
  firestore: {
    rules: fs.readFileSync("firestore.rules", "utf8"),
    host: "localhost",
    port: 8080,
  },
});

to this:

testEnv = await initializeTestEnvironment({
  projectId: PROJECT_ID,
  firestore: {
    rules: fs.readFileSync("firestore.rules", "utf8"),
    host: "127.0.0.1",
    port: 8080,
  },
});

[greg-md]

@gkawin thank you! It worked on node18 by adding NODE_OPTIONS=--dns-result-order=ipv4first.

[srtager555]

i can confirm that adding both of these blocks fixes the issue for us:

    "hub": {
      "host": "0.0.0.0",
      "port": 4400
    },
    "logging": {
      "host": "0.0.0.0",
      "port": 4500
    },

i can also confirm the exact same log output as above. running docker on mac and there's no ipv6 support.

I have the same issue but is when I use the firestore emulator with the sdk admin in a NextJS app on Mac and adding this to my firebase.json fix the issue