Dockerhub alerta-web 8.7.0 with Mongo 4.4.2 stops responding

[flinthamm]

Issue Summary Attempting to get Alerta working in Docker with Mongo rather than the more documented Postgres using "alerta/alerta-web:8.7.0" and "mongo:4.4.2" as the hardware has no AVX support and the latest Mongo 5 is only supported by pyMongo4 anyway.

The first hurdle was the Mongo database did not seem to be automatically created but this was resolved by using a mongo-init.js script to create a user with dbOwner privileges for a "monitoring" database. The DATABASE_URL environment in Alerta then successfully connected and initialised the "monitoring" database.

The next hurdle which I've still not managed to overcome is the "504 Gateway Timeout" which happens soon after both containers are up and running. The strange thing is sometimes, you can actually load the web interface for Alerta but this is at best and once you try to navigate to any of the menus it quickly just starts becoming unresponsive.

Environment

• OS: Linux

• API version: Latest DockerHub image (Dec'21 8.7.0 but have also tried 8.6.4)

• Deployment: Docker, self-hosted

• For self-hosted, WSGI environment: [eg. nginx/uwsgi, apache/mod_wsgi] ??

• Database: MongoDB 4.4.2 becuase I can't use 5 and no documentation specifies a specific version?

• Server config: Auth enabled? Yes & No - Have tried both Auth provider? Have only tried Basic Customer views? Default, so has not been specified ** (or provide the /config endpoint output) ** This part of course is critical but comments I've read in the "Issues" suggest the "/web/config.json" do not need to be mapped in docker-compose and the default one in the image which just configures "endpoint" as "/api" is sufficient? If you exec into the running docker image there is an "config.json.example" and this specified the endpoint as "http://localhost:8080" which has also been tried.

• web UI version: As per Docker Hub alerta-web 8.7.0 image

• CLI version: As per Docker Hub alerta-web 8.7.0 image

To Reproduce Simply follow all the instructions on the Docker Hub "https://registry.hub.docker.com/r/alerta/alerta-web" page, except use mongo image, rather than Postgres image e.g.

version: '2.1'
services:
  alerta-web:
    image: alerta/alerta-web:8.7.0
    ports:
      - "8180:8080"
    depends_on:
      - alerta-db
    volumes:
      - /docker/alerta/config.json:/web/config.json:ro
    environment:
      - DEBUG=1
#      - UWSGI_LISTEN=128 #This was added due to issue comments but did not help
      - DATABASE_URL=mongodb://alerta-db:27017/monitoring
      - DATABASE_NAME=monitoring
      - AUTH_REQUIRED=True
      - [email protected]
      - ADMIN_KEY=changeme
      - PLUGINS=reject,blackout,normalise,enhance
    restart: always
    
  alerta-db:
    image: mongo:4.4.2
    ports:
      - 27017:27017    
    restart: always
#    command: --smallfiles    
    volumes:
      - /docker/alerta-db:/data/db

Error: Failed to get config from http://localhost:8080/api/config. Reason: 504 Server Error: Gateway Time-out for url: http://localhost:8080/api/config

Expected behavior Having followed the documentation and tried a LOT of different docker-compose tweaks to Mongo and Alerta it feels like the only way forward is to ask. After trying different options for the config.json file i.e. No volume map at all (as per docker hub doc) and then using various options (see above) I finally found that giving the endpoint as the external docker interface i.e. http://192.168.x.x:8081/api" (where x.x. is the local IP and the modified port) seemed to be the solution.

However, although the web interface came up more reliably and some navigation of the menu interface would generally succeed, within a few more clicks the web interface stopped responding again! On subsequent rebuild of the container it again was unreliable with the web interface not getting passed the initial load.

As I've been writing this I've been doing more checks and seem to be getting closer to the problem. I thought setting the config.json endpoint to the external docker interface had solved the 504 Gateway Timeout issue but it seems like "http://localhost:8080/api/config" is being picked up from somewhere else and these error messages are still being generated? Presumably this endpoint is still valid from within the container though?

The main error message when the problem occurs seems to be: Error: Failed to get config from http://localhost:8080/api/config. Reason: 504 Server Error: Gateway Time-out for url: http://localhost:8080/api/config

Perhaps the main problem to solve comes from an earlier failure (just after debug output from the plugin Telegram) which I don't understand?

2022-02-16 10:59:24,340 DEBG 'uwsgi' stdout output:
unable to load app 0 (mountpoint='') (callable not found or import error)
mounting wsgi:app on /api

2022-02-16 10:59:24,340 DEBG 'uwsgi' stdout output:
WSGI app 0 (mountpoint='/api') ready in 0 seconds on interpreter 0x55cb04775470 pid: 69 (default app)
*** uWSGI is running in multiple interpreter mode ***

[GraFindlay]

Hi @flinthamm - this seems very similar to the issue I reported last week - https://github.com/alerta/docker-alerta/issues/374

I don't have a solution for this either.

[flinthamm]

Hi @GraFindlay and thanks for your reply. I did see your issue (#374) and perhaps should replied there instead but thought it was worth documenting the slightly different setup. Maybe the devs will see it differently but regardless think it requires them to merge the two?

Have you by chance tried a Postgres database instead of Mongo, as that seems to be a common factor and certainly the DockerHub and other examples I've found tend to always steer towards this setup which is perhaps more tested? I'm sure I read somewhere that Mongo was the primary database used during development though and postgres came later?

I've also conducted a few searches through the Slack channel but all suggestions so far have not improved the situation. I wonder if that is more active and we should really be posting there instead? The last suggestion that seemed to cure a similar problem (but not mine) was adding the environment UWSGI_LISTEN=128 to override the recently reduced value that now defaults to 100 having previously been 256? See https://github.com/alerta/docker-alerta/pull/344. I've not tried setting this value any higher and was something else I was going to research.

I'll be sure to post here if a fix or update becomes available and hopefully others are watching this and will do the same :)

[GraFindlay]

Hi @flinthamm I abandoned ny attempt to get Alerta up and running with Mongo in a docker container. However (good news) I managed to get Alerta working in docker with Postgres by following this tutorial: https://docs.alerta.io/gettingstarted/tutorial-10-docker.html#tutorial-10

[flinthamm]

Hi @GraFindlay I appreciate the update and glad you're now up and running. I've not given up but have run out of time for the moment - if and when this changes (possibly if and when there is a new docker release or more response) I'll be sure to post here. For this setup, my preference is Mongo.

[rabam]

I tried version 8.6.3. It seems to work with MongoDB. Unfortunately I do not know why. I guess it has nothing to do with the changes in this repo, but with the alerta repo. I tried version 8.6.3 with the master here. That did work.

[deus93]

I had the same issue, but I used mongo 5.0.6, alerta-web:8.7.0 (also tried 8.6.3). No errors about db connection in logs. fixed by switching to postgres db. Behavior: After the first start, everything works fine until the first uWSGI workers respawn.

ps. alerta-web:8.6.3 and mongo:4.4.3 are working fine.

[satterly]

Fixed by #435