Docker.Official.Image icon indicating copy to clipboard operation
Docker.Official.Image copied to clipboard

Can't connect to MongoDB

Open fatcatt013 opened this issue 2 years ago • 26 comments

MongoServerSelectionError: Server selection timed out after 30000 ms
    at Timeout._onTimeout (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/sdam/topology.js:312:38)
    at listOnTimeout (internal/timers.js:557:17)
    at processTimers (internal/timers.js:500:7) {
  reason: TopologyDescription {
    type: 'ReplicaSetNoPrimary',
    servers: Map(1) {
      'mongodb:27017' => ServerDescription {
        _hostAddress: HostAddress { isIPv6: false, host: 'mongodb', port: 27017 },
        address: 'mongodb:27017',
        type: 'RSGhost',
        hosts: [],
        passives: [],
        arbiters: [],
        tags: {},
        minWireVersion: 0,
        maxWireVersion: 9,
        roundTripTime: 11,
        lastUpdateTime: 809068,
        lastWriteDate: 0,
        topologyVersion: {
          processId: ObjectId { [Symbol(id)]: [Buffer [Uint8Array]] },
          counter: 0
        },
        logicalSessionTimeoutMinutes: 30
      }
    },
    stale: false,
    compatible: true,
    heartbeatFrequencyMS: 10000,
    localThresholdMS: 15,
    setName: 'rs0',
    commonWireVersion: 9,
    logicalSessionTimeoutMinutes: undefined
  }
}

I fresh installed with docker compose up -d and then it run.

RocketChat container gives me this error every 30 seconds and crashes. There are no other logs on that container.

{"t":{"$date":"2022-07-25T12:46:03.855+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.18.0.3:42482","connectionId":20,"connectionCount":2}}
{"t":{"$date":"2022-07-25T12:46:03.856+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn20","msg":"client metadata","attr":{"remote":"172.18.0.3:42482","client":"conn20","doc":{"driver":{"name":"nodejs","version":"4.3.1"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"5.10.104-linuxkit"},"platform":"Node.js v14.19.3, LE (unified)|Node.js v14.19.3, LE (unified)"}}}

In the MongoDB log I get this message that Connection from IP 172.18.0.3 (rocketChat) was accepted. I tried to reinstall the whole docker compose ecosystem even deleting and pulling the images again but it doesn't work.

fatcatt013 avatar Jul 25 '22 12:07 fatcatt013

https://github.com/RocketChat/Rocket.Chat/issues/26231

FIX

fatcatt013 avatar Jul 26 '22 14:07 fatcatt013

RocketChat/Rocket.Chat#26231

FIX

does not solve this problem, I added directConnection=true, with docker it probably solves it, without docker the error still remains mongodb 5.0

DeadMozay avatar Jul 27 '22 11:07 DeadMozay

yeah, I use docker compose..

still it is mongodb problem and should be solved with the code I provided.

fatcatt013 avatar Jul 27 '22 11:07 fatcatt013

No luck with that fix unfortunately

quasimodo-r avatar Jul 27 '22 12:07 quasimodo-r

❯ echo $MONGO_URL              
mongodb://127.0.0.1:27017/rocketchat?replicaSet=rs0&directConnection=true
❯ echo $MONGO_OPLOG_URL         
mongodb://127.0.0.1:27017/local?replicaSet=rs0&directConnection=true
❯ /usr/bin/node /opt/Rocket.Chat/main.js
/opt/Rocket.Chat/programs/server/node_modules/fibers/future.js:313
						throw(ex);
						^

MongoServerSelectionError: Server selection timed out after 30000 ms
    at Timeout._onTimeout (/opt/Rocket.Chat/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/sdam/topology.js:312:38)
    at listOnTimeout (internal/timers.js:557:17)
    at processTimers (internal/timers.js:500:7) {
  reason: TopologyDescription {
    type: 'Single',
    servers: Map(1) {
      '127.0.0.1:27017' => ServerDescription {
        _hostAddress: HostAddress { isIPv6: false, host: '127.0.0.1', port: 27017 },
        address: '127.0.0.1:27017',
        type: 'Unknown',
        hosts: [],
        passives: [],
        arbiters: [],
        tags: {},
        minWireVersion: 0,
        maxWireVersion: 0,
        roundTripTime: -1,
        lastUpdateTime: 477254,
        lastWriteDate: 0
      }
    },
    stale: false,
    compatible: true,
    heartbeatFrequencyMS: 10000,
    localThresholdMS: 15,
    setName: 'rs0',
    logicalSessionTimeoutMinutes: undefined
  }
}
db version v5.0.9
Build Info: {
    "version": "5.0.9",
    "gitVersion": "6f7dae919422dcd7f4892c10ff20cdc721ad00e6",
    "openSSLVersion": "OpenSSL 1.1.1l  24 Aug 2021 SUSE release 150400.7.7.1",
    "modules": [],
    "allocator": "tcmalloc",
    "environment": {
        "distmod": "suse15",
        "distarch": "x86_64",
        "target_arch": "x86_64"
    }
}
❯ /usr/bin/node --version               
v14.20.0

DeadMozay avatar Jul 27 '22 13:07 DeadMozay

Try this

shell into mongo and write

rs.initiate( _id: "rs0", members: [ { _id: 0, host: "MONGODB URL" } ] )

fatcatt013 avatar Jul 27 '22 13:07 fatcatt013

rs.initiate( _id: "rs0", members: [ { _id: 0, host: "MONGODB URL" } ] )

already done, did not help

replocal:PRIMARY> rs.initiate({_id: "rs0", members: [{ _id: 0, host : "localhost:27017" }]})
{
	"ok" : 0,
	"errmsg" : "already initialized",
	"code" : 23,
	"codeName" : "AlreadyInitialized",
	"$clusterTime" : {
		"clusterTime" : Timestamp(1658930022, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	},
	"operationTime" : Timestamp(1658930022, 1)
}

DeadMozay avatar Jul 27 '22 13:07 DeadMozay

Ok, drop the database beforehand, then use rs.initiate()

fatcatt013 avatar Jul 27 '22 14:07 fatcatt013

and it will not delete all data from a DB?

DeadMozay avatar Jul 27 '22 14:07 DeadMozay

It will.. you completely reset it.. but I assume that is not the biggest problem now if the whole db is not working anyway

fatcatt013 avatar Jul 27 '22 14:07 fatcatt013

it works up to rocketchat version 5.0.0

❯ mongo rocketchat --eval "db.dropDatabase()"                                                         
MongoDB shell version v5.0.9
connecting to: mongodb://127.0.0.1:27017/rocketchat?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("a2f03843-1f72-421d-afdc-3e0259c8d21f") }
MongoDB server version: 5.0.9
{
	"ok" : 1,
	"$clusterTime" : {
		"clusterTime" : Timestamp(1658932730, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	},
	"operationTime" : Timestamp(1658932730, 1)
}
❯ mongo rocketchat --eval 'rs.initiate({_id: "rs0", members: [{ _id: 0, host : "localhost:27017" }]})'
MongoDB shell version v5.0.9
connecting to: mongodb://127.0.0.1:27017/rocketchat?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("9e60954a-bb80-4ec7-9453-c1e9b607eb68") }
MongoDB server version: 5.0.9
{
	"ok" : 0,
	"errmsg" : "already initialized",
	"code" : 23,
	"codeName" : "AlreadyInitialized",
	"$clusterTime" : {
		"clusterTime" : Timestamp(1658932790, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	},
	"operationTime" : Timestamp(1658932790, 1)
}

why is it still an error already initialized

DeadMozay avatar Jul 27 '22 14:07 DeadMozay

because you try to initialize replica which is already initialized, thats why you do the reconfig() option I linked above.

fatcatt013 avatar Jul 27 '22 14:07 fatcatt013

reconfig() did not help Fixed, dropped the local database, performed initialization, restored from a backup, it worked

DeadMozay avatar Jul 28 '22 03:07 DeadMozay

I have exactly the same problem. Trying (and failing) with

      - MONGO_URL=mongodb://mongo:27017/rocketchat_${NAME}?directConnection=true
      - MONGO_OPLOG_URL=mongodb://mongo:27017/local?replicaSet=rs01&directConnection=true

Before upgrading to 5.0, this worked:

      - MONGO_URL=mongodb://mongo:27017/rocketchat_${NAME}
      - MONGO_OPLOG_URL=mongodb://mongo:27017/local?replSet=rs01

steffen-kdab avatar Jul 28 '22 09:07 steffen-kdab

I guess this needs to be updated for 5.0? https://docs.rocket.chat/quick-start/environment-configuration/mongodb-configuration/mongo-replicas

steffen-kdab avatar Jul 28 '22 13:07 steffen-kdab

Hey everyone, can someone show me the compose file you're using?

debdutdeb avatar Aug 04 '22 10:08 debdutdeb

In my case it also helped to add &directConnection=true to the MONGO_URL and MONGO_OPLOG_URL, like its mentioned in the release. But shouldn't it be in the compose.yml then?

HyP3r- avatar Aug 06 '22 14:08 HyP3r-

You don't need that in this compose file.

debdutdeb avatar Aug 07 '22 00:08 debdutdeb

Maybe you misunderstood me. If you leave out &directConnection=true it does not work. So in this repository, in this project, shouldn't the official compose.yml of Rocket.Chat be adjusted if this is necessary?

If this is only necessary for some configurations there should be a switch in the env.example file. Not everyone who deals with all this has the time and nerve to read everything in such detail.

HyP3r- avatar Aug 07 '22 17:08 HyP3r-

I'm sorry. You only need that if you're using the old docker compose file (one that doesn't use bitnami image). Otherwise, can you please share the file you're using? So that I can take a look.

debdutdeb avatar Aug 07 '22 18:08 debdutdeb

If this is only necessary for some configurations there should be a switch in the env.example file.

As I mentioned already, it is only required if someone is using the old compose file. Not the one that is sitting in this repository today. If indeed that is not the case I'm asking for someone to share their compose file here.

Not everyone who deals with all this has the time and nerve to read everything in such detail.

If directConnection is working for someone then that's fine and they can keep on using that configuration. My purpose of commenting is to make sure there is no misunderstanding that the new template needs the parameter. It doesn't.

debdutdeb avatar Aug 07 '22 18:08 debdutdeb

Here are the two files in my environment:

.env:

### Rocket.Chat configuration

# Rocket.Chat version
# see:- https://github.com/RocketChat/Rocket.Chat/releases
#RELEASE=
# MongoDB endpoint (include ?replicaSet= parameter)
MONGO_URL=mongodb://mongodb:27017/rocketchat?replicaSet=rs0&directConnection=true
# MongoDB endpoint to the local database
MONGO_OPLOG_URL=mongodb://mongodb:27017/local?replicaSet=rs0&directConnection=true
# IP to bind the process to
BIND_IP=127.0.0.1
# URL used to access your Rocket.Chat instance
ROOT_URL=https://a.company.in.the.world.com
# Port Rocket.Chat runs on (in-container)
PORT=3000
# Port on the host to bind to
HOST_PORT=3011

### MongoDB configuration
# MongoDB version/image tag
#MONGODB_RELEASE=
# See:- https://hub.docker.com/r/bitnami/mongodb

### Traefik config (if enabled)
# Traefik version/image tag
#TRAEFIK_RELEASE=
# Domain for https (change ROOT_URL & BIND_IP accordingly)
#DOMAIN=
# Email for certificate notifications
#LETSENCRYPT_EMAIL=

docker-compose.yml:

version: "3.7"

services:
    rocketchat:
        image: rocket.chat:${RELEASE:-latest}
        restart: unless-stopped
        environment:
            MONGO_URL: "${MONGO_URL:-mongodb://${MONGODB_ADVERTISED_HOSTNAME:-mongodb}:${MONGODB_INITIAL_PRIMARY_PORT_NUMBER:-27017}/${MONGODB_DATABASE:-rocketchat}?replicaSet=${MONGODB_REPLICA_SET_NAME:-rs0}}"
            MONGO_OPLOG_URL: "${MONGO_OPLOG_URL:-mongodb://${MONGODB_ADVERTISED_HOSTNAME:-mongodb}:${MONGODB_INITIAL_PRIMARY_PORT_NUMBER:-27017}/local?replicaSet=${MONGODB_REPLICA_SET_NAME:-rs0}}"
            ROOT_URL: ${ROOT_URL:-http://localhost:${HOST_PORT:-3000}}
            PORT: ${PORT:-3000}
            DEPLOY_METHOD: docker
            DEPLOY_PLATFORM: ${DEPLOY_PLATFORM}
        depends_on:
            - mongo
        ports:
            - "${BIND_IP:-0.0.0.0}:${HOST_PORT:-3000}:${PORT:-3000}"

    mongodb:
        image: docker.io/bitnami/mongodb:${MONGODB_VERSION:-4.4}
        restart: unless-stopped
        volumes:
            - mongo-db:/bitnami/mongodb
        environment:
            MONGODB_REPLICA_SET_MODE: primary
            MONGODB_REPLICA_SET_NAME: ${MONGODB_REPLICA_SET_NAME:-rs0}
            MONGODB_PORT_NUMBER: ${MONGODB_PORT_NUMBER:-27017}
            MONGODB_INITIAL_PRIMARY_HOST: ${MONGODB_INITIAL_PRIMARY_HOST:-mongodb}
            MONGODB_INITIAL_PRIMARY_PORT_NUMBER: ${MONGODB_INITIAL_PRIMARY_PORT_NUMBER:-27017}
            MONGODB_ADVERTISED_HOSTNAME: ${MONGODB_ADVERTISED_HOSTNAME:-mongodb}
            MONGODB_ENABLE_JOURNAL: ${MONGODB_ENABLE_JOURNAL:-true}
            ALLOW_EMPTY_PASSWORD: ${ALLOW_EMPTY_PASSWORD:-yes}

volumes:
    mongo-db: { driver: local }

As you can see I'm using the current version of the compose. I just have altered some parts like the volumes and the image of the rockat.chat (registry.rocket.chat/rocketchat/rocket.chat:${RELEASE:-latest}) itself. And if I comment out the lines MONGO_URL and MONGO_OPLOG_URL in my .env file, the system is not working!

Like the author of this issue, I then have this error:

❯ /usr/bin/node /opt/Rocket.Chat/main.js
/opt/Rocket.Chat/programs/server/node_modules/fibers/future.js:313
						throw(ex);

HyP3r- avatar Aug 07 '22 18:08 HyP3r-

Like the author of this issue, I then have this error:

I can't see the whole error :(

Couple questions:

  1. Did you migrate from an older install ? (basically changed thecompose file?)
  2. What is your docker compose version?
  3. Can you show us part of your rs config? Log into the mongodb container, in the mongo shell execute
rs.config().members.map(member => member.host)

Thanks a lot for helping in detecting the problem!

debdutdeb avatar Aug 07 '22 19:08 debdutdeb

Managed to fix our issue with the cfg = rs.conf() cfg.members[0].host = "mongo:27017" rs.reconfig(cfg)

and then stopping the container and removing the replica conf alltogether from docker-compose.yml + and then restarting.

quasimodo-r avatar Aug 08 '22 04:08 quasimodo-r

Like the author of this issue, I then have this error:

I can't see the whole error :(

Yeah thats true. If you want I can post the full output.

Couple questions:

1. Did you migrate from an older install ? (basically changed thecompose file?)

Yes. I have updated the compose file because the old one was also not working with the new rocket.chat version.

2. What is your docker compose version?

Here my docker compose version:

$ docker compose version
Docker Compose version v2.6.0
3. Can you show us part of your rs config?
   Log into the mongodb container, in the mongo shell execute
rs.config().members.map(member => member.host)

Here the output:

rs0:PRIMARY> rs.config().members.map(member => member.host)
[ "localhost:27017" ]

What I didn't know is that Rocket.Chat is storing the connection configuration in the mongodb. So Rocket.Chat is connecting to mongodb and then is connecting to the target mongodb. Like a bootstrap. As I said: I didn't know.

So like @quasimodo-r posted the solution was also for me:

cfg = rs.conf()
cfg.members[0].host = "mongo:27017"
rs.reconfig(cfg)

And as I also said: this should be mentioned somewhere! I don't have the time to start a Code-Behind reading to troubleshoot such issues.

HyP3r- avatar Aug 09 '22 22:08 HyP3r-

Hello,

I'm following this issue since i have done all the steps above and i have an error that i can't find online solution: When i do the final steps explain above :

cfg = rs.conf()
cfg.members[0].host = "mongo:27017"
rs.reconfig(cfg)

I got this error message from mongo

> rs.reconfig(cfg)

{

	"topologyVersion" : {

		"processId" : ObjectId("63ff1094e1a3ebd2a31dcecb"),

		"counter" : NumberLong(1)

	},

	"ok" : 0,

	"errmsg" : "New config is rejected :: caused by :: replSetReconfig should only be run on a writable PRIMARY. Current state REMOVED;",

	"code" : 10107,

	"codeName" : "NotWritablePrimary"

Can you help me with that?

EDIT: It seems like i'm not connected to the mongo db with the master privileges:

> show dbs

uncaught exception: Error: listDatabases failed:{

	"topologyVersion" : {

		"processId" : ObjectId("63ff150247d2239ee2a76a7d"),

		"counter" : NumberLong(1)

	},

	"ok" : 0,

	"errmsg" : "not master and slaveOk=false",

	"code" : 13435,

	"codeName" : "NotPrimaryNoSecondaryOk"

} :

_getErrorWithCode@src/mongo/shell/utils.js:25:13

Mongo.prototype.getDBs/<@src/mongo/shell/mongo.js:147:19

Mongo.prototype.getDBs@src/mongo/shell/mongo.js:99:12

shellHelper.show@src/mongo/shell/utils.js:937:13

shellHelper@src/mongo/shell/utils.js:819:15

@(shellhelp2):1:1

But i haven't specify any user in my compose.yml file

compose.yml:

services:

  rocketchat:

    image: registry.rocket.chat/rocketchat/rocket.chat:${RELEASE:-5.3}

    restart: on-failure

    labels:

      traefik.enable: "true"

      traefik.http.routers.rocketchat.rule: Host(`${DOMAIN}`)

      traefik.http.routers.rocketchat.tls: "true"

      traefik.http.routers.rocketchat.entrypoints: https

      traefik.http.routers.rocketchat.tls.certresolver: le

    environment:

      MONGO_URL: ${MONGO_URL:-mongodb://mongodb:27017/rocketchat?replicaSet=rs0}

      MONGO_OPLOG_URL: ${MONGO_URL:-mongodb://mongodb:27017/local?replicaSet=rs0}

      ROOT_URL: ${ROOT_URL:-http://localhost:${HOST_PORT:-3000}}

      PORT: ${PORT:-3000}

      DEPLOY_METHOD: docker

      DEPLOY_PLATFORM: ${DEPLOY_PLATFORM}

    depends_on:

      - mongodb

    expose:

      - ${PORT:-3000}

    ports:

      - "${BIND_IP:-0.0.0.0}:${HOST_PORT:-3000}:${PORT:-3000}"



  mongodb:

    image: docker.io/bitnami/mongodb:${MONGODB_VERSION:-4.4}

    restart: on-failure

    volumes:

      - mongodb_data:/bitnami/mongodb

    ports:

      - "27017:27017"

    environment:

      MONGODB_REPLICA_SET_MODE: primary

      MONGODB_REPLICA_SET_NAME: ${MONGODB_REPLICA_SET_NAME:-rs0}

      MONGODB_PORT_NUMBER: ${MONGODB_PORT_NUMBER:-27017}

      MONGODB_INITIAL_PRIMARY_HOST: ${MONGODB_INITIAL_PRIMARY_HOST:-mongodb}

      MONGODB_INITIAL_PRIMARY_PORT_NUMBER: ${MONGODB_INITIAL_PRIMARY_PORT_NUMBER:-27017}

      MONGODB_ADVERTISED_HOSTNAME: ${MONGODB_ADVERTISED_HOSTNAME:-mongodb}

      MONGODB_ENABLE_JOURNAL: ${MONGODB_ENABLE_JOURNAL:-true}

      ALLOW_EMPTY_PASSWORD: ${ALLOW_EMPTY_PASSWORD:-yes}



volumes:

  mongodb_data: { driver: local }

I followed another github issue to hard codded the MONGO_URL and the MONGO_OPLOG_URL.

DatraX777 avatar Mar 01 '23 08:03 DatraX777