docker icon indicating copy to clipboard operation
docker copied to clipboard

Containers keep crashing and spam logs

Open J0J0HA opened this issue 1 year ago • 2 comments

When I run docker compose up, at first everything looks like it would be going well, but then, after like 10-15secs, multiple containers stop at once with the error: MongoServerSelectionError: connection timed out. These are bbb-docker-html5-frontend-1-1, bbb-docker-html5-backend-1-1, bbb-docker-html5-frontend-2-1 and bbb-docker-html5-backend-2-1.

At the same time, bbb-docker-greenlight-1 writes Waiting for postgres to start up ... over and over again. (postgres is already running successfully at that moment)

Additionally, bbb-docker-bbb-web-1 crashes with this:

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'slidesGenerationProgressNotifier' defined in class path resource [spring/doc-conversion.xml]: Cannot resolve reference to bean 'bbbWebApiGWApp' while setting bean property 'messagingService'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'bbbWebApiGWApp' defined in class path resource [spring/resources.xml]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.bigbluebutton.api2.BbbWebApiGWApp]: Constructor threw exception; nested exception is io.lettuce.core.RedisConnectionException: Unable to connect to redis/<unresolved>:6379

bbb-docker-webrtc-sfu-1 repetedly says {"errorMessage":"Redis connection to 10.7.7.5:6379 failed - connect EHOSTUNREACH 10.7.7.5:6379","label":"screenshare","level":"error","message":"Redis client error event","timestamp":"2024-01-04T14:24:50.050Z"

bbb-docker-recordings-1 says:

rake aborted!
Redis::CannotConnectError: Error connecting to Redis on redis:6379 (Redis::TimeoutError)

(plus traceback)

bbb-docker-etherpad-1 says:

[2024-01-04 14:24:45.679] [ERROR] server - Error occurred while starting Etherpad
[2024-01-04 14:24:45.679] [ERROR] server - Metrics at time of fatal error:
{
  "httpStartTime": 0,
  "memoryUsage": 93655040,
  "memoryUsageHeap": 25224256
}
[2024-01-04 14:24:45.680] [ERROR] server - Error: Connection timeout
[2024-01-04 14:24:45.680] [INFO] server - Exiting...
[2024-01-04 14:24:45.680] [INFO] server - Waiting for Node.js to exit...
[2024-01-04 14:24:50.681] [ERROR] server - Something that should have been cleaned up during the shutdown hook (such as a timer, worker thread, or open connection) is preventing Node.js from exiting
[2024-01-04 14:24:50.681] [ERROR] server - Enable `dumpOnUncleanExit` setting to get a dump of objects preventing a clean exit
[2024-01-04 14:24:50.681] [ERROR] server - Forcing an unclean exit...

Some other containers also crash because of not being able to connect to redis or mongodb, although I have not listed them here.

My server fullfills all requirements:

  • I have more then 4GM free RAM
  • Is running on Ubuntu (-> Linux)
  • I have tried runnigng it with sudo and also running it with the root user itself, no success.
  • I have a Public IPv4
  • docker was installed using the documentation attached link (get.docker.com)
  • Docker works fine with everything else
  • docker version is 24.0.7 build afdd53b

I am using the main branch and followed the steps to integrate in an existing web server.

Full logs: https://gist.github.com/J0J0HA/1c2161e31b19cf18c27280948df4d63d

J0J0HA avatar Jan 04 '24 14:01 J0J0HA

I guess the databases (postgres & redis) themself are running? see docker compose ps

then it sounds like networking issue, most likely a firewall prevents the containers from accessing each other.

As far as I know ufw comes now preinstalled & active on ubuntu setups, which requires...

$ ufw allow from 10.7.7.0/24

alangecker avatar Jan 06 '24 16:01 alangecker

I checked with docker ps and postgres and redis are running. I added the ufw rule like you recommended, but it still does not work (same errors as before). (ufw was disabled, as I noticed afterwards).

Actually it should not have any firewalls installed, as it has an external firewall (it is running on a VPS).

J0J0HA avatar Jan 09 '24 17:01 J0J0HA