kontena-stacks
kontena-stacks copied to clipboard
Peers shutting down unexpectedly
I'm having some issues with this stack. For some reason my peers keep shutting down for no apparent reason. Here's the logs from the last 2 peers.
https://hastebin.com/cesalapede.vbs
Why do they keep shutting down ?
Maybe it's because the startup command is missing the fork parameter ?
https://docs.mongodb.com/manual/tutorial/manage-mongodb-processes/#start-mongod-as-a-daemon
Here's the startup options.. Shouldn't there be a fork parameter ?
2018-01-13T15:57:09.055+0000 I CONTROL [initandlisten] options: { net: { bindIp: "0.0.0.0" }, replication: { replSet: "mongo-rs" }, security: { keyFile: "/data/db/mongo.key" }, storage: { wiredTiger: { engineConfig: { cacheSizeGB: 0.5 } } } }
Should there not be a --fork in this command ?
https://github.com/kontena/kontena-stacks/blob/master/mongodb-replica-set/kontena.yml#L35
2 of my nodes just did it again.. Still havn't found the issue..
2018-02-03T06:36:42.316+0000 I CONTROL [signalProcessingThread] got signal 15 (Terminated), will terminate after current cmd ends
2018-02-03T06:36:42.316+0000 I NETWORK [signalProcessingThread] shutdown: going to close listening sockets...
2018-02-03T06:36:42.316+0000 I NETWORK [signalProcessingThread] removing socket file: /tmp/mongodb-27017.sock
2018-02-03T06:36:42.402+0000 I REPL [signalProcessingThread] shutting down replication subsystems
2018-02-03T06:36:42.402+0000 I REPL [signalProcessingThread] Stopping replication reporter thread
2018-02-03T06:36:42.402+0000 I REPL [signalProcessingThread] Stopping replication fetcher thread
2018-02-03T06:36:42.402+0000 I REPL [signalProcessingThread] Stopping replication applier thread
2018-02-03T06:36:42.873+0000 I REPL [signalProcessingThread] Stopping replication storage threads
2018-02-03T06:36:42.893+0000 I FTDC [signalProcessingThread] Shutting down full-time diagnostic data capture
2018-02-03T06:36:42.900+0000 I STORAGE [WTOplogJournalThread] oplog journal thread loop shutting down
2018-02-03T06:36:42.901+0000 I STORAGE [signalProcessingThread] WiredTigerKVEngine shutting down
2018-02-03T06:36:44.682+0000 I STORAGE [signalProcessingThread] shutdown: removing fs lock...
2018-02-03T06:36:44.682+0000 I CONTROL [signalProcessingThread] now exiting
2018-02-03T06:36:44.683+0000 I CONTROL [signalProcessingThread] shutting down with code:0
I face the same issue, but i have only two nodes (replicaSet), one node works well (primary), but slave is shutting down time to time. Time is different, and it may be 1 week or several days from starting mongodb server. I start server by issuing command: /etc/init.d/mongodb start After start I close terminal. Thus if it was a cause of shutting down mongodb it would be in time close to exiting from terminal, but it can got terminate signal after week of work.
mongodb is running on Ubuntu 14.04.5 LTS
My starting log:
[main] ***** SERVER RESTARTED ***** 2020-03-01T07:29:25.010+0000 I CONTROL [initandlisten] MongoDB starting : pid=21266 port=27017 dbpath=/home3/mongodb 64-bit host=127.0.0.1 2020-03-01T07:29:25.010+0000 I CONTROL [initandlisten] db version v3.2.9 2020-03-01T07:29:25.010+0000 I CONTROL [initandlisten] git version: 22ec9e93b40c85fc7cae7d56e7d6a02fd811088c 2020-03-01T07:29:25.010+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1f 6 Jan 2014 2020-03-01T07:29:25.010+0000 I CONTROL [initandlisten] allocator: tcmalloc 2020-03-01T07:29:25.010+0000 I CONTROL [initandlisten] modules: none 2020-03-01T07:29:25.010+0000 I CONTROL [initandlisten] build environment: 2020-03-01T07:29:25.010+0000 I CONTROL [initandlisten] distmod: ubuntu1404 2020-03-01T07:29:25.010+0000 I CONTROL [initandlisten] distarch: x86_64 2020-03-01T07:29:25.010+0000 I CONTROL [initandlisten] target_arch: x86_64 2020-03-01T07:29:25.010+0000 I CONTROL [initandlisten] options: { config: "/etc/mongodb.conf", net: { bindIp: "0.0.0.0", port: 27017 }, replication: { replSet: "tabsbook" }, security: { authorization: "enabled", keyFile: "/opt/dockers/mongodb/mongodb-keyfile" }, storage: { dbPath: "/home3/mongodb", journal: { enabled: true }, wiredTiger: { engineConfig: { cacheSizeGB: 1 } } }, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongodb.log" } }