WebODM

WebODM copied to clipboard

How did you install WebODM (docker, installer, etc.)?

Linux docker installation throuhg webodm.sh start

What's your browser and operating system? (Copy/paste the output of https://www.whatismybrowser.com/)

Linux Ubuntu Jammy / Lunar

What is the problem?

WebODM installs correctly, but refuses to run, due to Redis not lauching, reporting a broken install ( most probably redis crashes ). Same behaviour on Ubuntu Jammy and on Lunar

How can we reproduce this?

• Install fresh Ubuntu Jammy or Lunar and required dependencies for WebODM ( git, docker.io…)
• Follow WebODM installation instructions ( git clone & webodm.sh start )
• WebODM does not start

Error is exactly like the one reported here : https://community.opendronemap.org/t/broker-in-docker-cant-start-when-starting-over-git-bash/16416

---

 ./webodm.sh start
Checking for docker...   OK
Checking for docker-compose...  OK
Starting WebODM...

Using the following environment:
================================
Host: localhost
Port: 8000
Media directory: appmedia
Postgres DB directory: dbdata
SSL: NO
SSL key:
SSL certificate:
SSL insecure port redirect: 80
Celery Broker: redis://broker
Default Nodes: 1
================================
Make sure to issue a ./webodm.sh down if you decide to change the environment.

docker-compose -f docker-compose.yml -f docker-compose.nodeodm.yml up --scale node-odm=1
Starting db                ... done
Starting webodm_node-odm_1 ... done
Starting broker            ... done
Starting worker            ... done
Starting webapp            ... done
Attaching to db, webodm_node-odm_1, broker, worker, webapp
broker      | 1:C 15 Jun 2023 16:52:22.738 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
broker      | 1:C 15 Jun 2023 16:52:22.738 # Redis version=7.0.11, bits=64, commit=00000000, modified=0, pid=1, just started
broker      | 1:C 15 Jun 2023 16:52:22.738 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
broker      | 1:M 15 Jun 2023 16:52:22.738 * monotonic clock: POSIX clock_gettime
broker      | 1:M 15 Jun 2023 16:52:22.739 * Running mode=standalone, port=6379.
broker      | 1:M 15 Jun 2023 16:52:22.739 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
broker      | 1:M 15 Jun 2023 16:52:22.739 # Server initialized
broker      | 1:M 15 Jun 2023 16:52:22.739 # Fatal: Can't initialize Background Jobs.
db          |
db          | PostgreSQL Database directory appears to contain a database; Skipping initialization
db          |
db          | LOG:  database system was shut down at 2023-06-15 16:45:08 UTC
db          | LOG:  MultiXact member wraparound protections are now enabled
db          | LOG:  database system is ready to accept connections
db          | LOG:  autovacuum launcher started
node-odm_1  | (node:1) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.
node-odm_1  |
node-odm_1  | Please migrate your code to use AWS SDK for JavaScript (v3).
node-odm_1  | For more information, check the migration guide at https://a.co/7PzMCcy
node-odm_1  | (Use `node --trace-warnings ...` to show where the warning was created)
node-odm_1  | info: Authentication using NoTokenRequired
node-odm_1  | info: Listening on 0.0.0.0:6367 UDP for progress updates
node-odm_1  | info: Initialized 0 tasks
node-odm_1  | info: Checking for orphaned directories to be removed...
worker      | ldd: error: you do not have read permission for `/usr/lib/postgresql/13/bin/psql'
worker      | Use of uninitialized value $lib_path in concatenation (.) or string at /usr/bin/psql line 149.
node-odm_1  | info: Server has started on port 3000
worker      | Postgres is up - executing command
worker      | wait-for-it.sh: waiting for broker:6379 without a timeout
webapp      | ldd: error: you do not have read permission for `/usr/lib/postgresql/13/bin/psql'
webapp      | Use of uninitialized value $lib_path in concatenation (.) or string at /usr/bin/psql line 149.
webapp      | Postgres is up - executing command
webapp      | wait-for-it.sh: waiting for broker:6379 without a timeout
broker exited with code 1
webapp      | wait-for-it.sh: broker:6379 is available after 1 seconds
webapp      |
webapp      |  _       __     __    ____  ____  __  ___
webapp      | | |     / /__  / /_  / __ \/ __ \/  |/  /
webapp      | | | /| / / _ \/ __ \/ / / / / / / /|_/ /
webapp      | | |/ |/ /  __/ /_/ / /_/ / /_/ / /  / /
webapp      | |__/|__/\___/_.___/\____/_____/_/  /_/
webapp      |
webapp      |
webapp      | Checking python version... 3.x, good!
webapp      | Checking GDAL version... GDAL 3.2.2, released 2021/03/05, excellent!
webapp      | Running migrations
webapp      | Operations to perform:
webapp      |   Apply all migrations: admin, app, auth, contenttypes, guardian, nodeodm, sessions
webapp      | Running migrations:
webapp      |   No migrations to apply.
broker exited with code 1
webapp      | Checking for celery...   OK
webapp      | Scheduler has shutdown.
webapp      | Generating nginx configurations from templates...
webapp      | - nginx/nginx-ssl.conf
webapp      | - nginx/nginx.conf
webapp      | celery beat v4.4.0 (cliffs) is starting.
broker exited with code 1
webapp      | [2023-06-15 16:52:28 +0000] [125] [INFO] Starting gunicorn 19.7.1
webapp      | [2023-06-15 16:52:28 +0000] [125] [INFO] Listening at: unix:/tmp/gunicorn.sock (125)
webapp      | [2023-06-15 16:52:28 +0000] [125] [INFO] Using worker: sync
webapp      | /usr/lib/python3.9/os.py:1023: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
webapp      |   return io.open(fd, *args, **kwargs)
webapp      | [2023-06-15 16:52:28 +0000] [168] [INFO] Booting worker with pid: 168
webapp      | [2023-06-15 16:52:28 +0000] [169] [INFO] Booting worker with pid: 169
webapp      | [2023-06-15 16:52:28 +0000] [170] [INFO] Booting worker with pid: 170
webapp      | [2023-06-15 16:52:28 +0000] [171] [INFO] Booting worker with pid: 171
webapp      | __    -    ... __   -        _
webapp      | LocalTime -> 2023-06-15 16:52:28
webapp      | Configuration ->
webapp      |     . broker -> redis://broker:6379//
webapp      |     . loader -> celery.loaders.app.AppLoader
webapp      |     . scheduler -> celery.beat.PersistentScheduler
webapp      |     . db -> celerybeat-schedule
webapp      |     . logfile -> [stderr]@%WARNING
webapp      |     . maxinterval -> 5.00 minutes (300s)
webapp      | [2023-06-15 16:52:28 +0000] [172] [INFO] Booting worker with pid: 172
webapp      | [2023-06-15 16:52:28 +0000] [173] [INFO] Booting worker with pid: 173
webapp      | [2023-06-15 16:52:28 +0000] [174] [INFO] Booting worker with pid: 174
webapp      | [2023-06-15 16:52:28 +0000] [175] [INFO] Booting worker with pid: 175
webapp      | [2023-06-15 16:52:28 +0000] [176] [INFO] Booting worker with pid: 176
webapp      | [2023-06-15 16:52:28 +0000] [177] [INFO] Booting worker with pid: 177
webapp      | [2023-06-15 16:52:28 +0000] [178] [INFO] Booting worker with pid: 178
webapp      | [2023-06-15 16:52:28 +0000] [179] [INFO] Booting worker with pid: 179
webapp      | [2023-06-15 16:52:28 +0000] [180] [INFO] Booting worker with pid: 180
webapp      | [2023-06-15 16:52:28 +0000] [181] [INFO] Booting worker with pid: 181
webapp      | [2023-06-15 16:52:28 +0000] [182] [INFO] Booting worker with pid: 182
webapp      | [2023-06-15 16:52:29 +0000] [183] [INFO] Booting worker with pid: 183
webapp      | [2023-06-15 16:52:29 +0000] [184] [INFO] Booting worker with pid: 184
webapp      | [2023-06-15 16:52:29 +0000] [185] [INFO] Booting worker with pid: 185
webapp      | [2023-06-15 16:52:29 +0000] [186] [INFO] Booting worker with pid: 186
webapp      | [2023-06-15 16:52:29 +0000] [187] [INFO] Booting worker with pid: 187
webapp      | [2023-06-15 16:52:29 +0000] [188] [INFO] Booting worker with pid: 188
webapp      | [2023-06-15 16:52:29 +0000] [189] [INFO] Booting worker with pid: 189
webapp      | [2023-06-15 16:52:29 +0000] [190] [INFO] Booting worker with pid: 190
webapp      | [2023-06-15 16:52:29 +0000] [191] [INFO] Booting worker with pid: 191
webapp      | [2023-06-15 16:52:29 +0000] [192] [INFO] Booting worker with pid: 192
broker exited with code 1
webapp      |
webapp      | Trying to establish communication...
webapp      | INFO Booting WebODM 2.0.3
webapp      | INFO Registered [coreplugins.lightning.plugin]
webapp      | INFO Registered [coreplugins.measure.plugin]
webapp      | INFO Registered [coreplugins.diagnostic.plugin]
webapp      | INFO Registered [coreplugins.contours.plugin]
webapp      | INFO Registered [coreplugins.fullscreen.plugin]
webapp      | INFO Registered [coreplugins.posm-gcpi.plugin]
webapp      | ERROR Cannot connect to celery broker at redis://broker. Make sure that your redis-server is running at that address: Error -5 connecting to broker:6379. No address associated with hostname.
webapp      |
webapp      | Congratulations! ?@(???)@?
webapp      | ==========================
webapp      |
webapp      | If there are no errors, WebODM should be up and running!
webapp      |
webapp      | Open a web browser and navigate to http://localhost:8000
webapp      |
webapp      | [2023-06-15 16:52:33,323: ERROR/MainProcess] beat: Connection error: Error -5 connecting to broker:6379. No address associated with hostname.. Trying again in 2.0 seconds...
webapp      | [2023-06-15 16:52:35,329: ERROR/MainProcess] beat: Connection error: Error -5 connecting to broker:6379. No address associated with hostname.. Trying again in 4.0 seconds...
broker exited with code 1
broker      | 1:C 15 Jun 2023 16:27:21.243 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
broker      | 1:C 15 Jun 2023 16:27:21.243 # Redis version=7.0.11, bits=64, commit=00000000, modified=0, pid=1, just started
broker      | 1:C 15 Jun 2023 16:27:21.243 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
broker      | 1:M 15 Jun 2023 16:27:21.243 * monotonic clock: POSIX clock_gettime
broker      | 1:M 15 Jun 2023 16:27:21.244 * Running mode=standalone, port=6379.
broker      | 1:M 15 Jun 2023 16:27:21.244 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
broker      | 1:M 15 Jun 2023 16:27:21.244 # Server initialized
broker      | 1:M 15 Jun 2023 16:27:21.244 # Fatal: Can't initialize Background Jobs.
broker      | 1:C 15 Jun 2023 16:27:21.707 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
broker      | 1:C 15 Jun 2023 16:27:21.707 # Redis version=7.0.11, bits=64, commit=00000000, modified=0, pid=1, just started
broker      | 1:C 15 Jun 2023 16:27:21.707 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
broker      | 1:M 15 Jun 2023 16:27:21.708 * monotonic clock: POSIX clock_gettime
broker      | 1:M 15 Jun 2023 16:27:21.708 * Running mode=standalone, port=6379.
broker      | 1:M 15 Jun 2023 16:27:21.708 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
broker      | 1:M 15 Jun 2023 16:27:21.708 # Server initialized
broker      | 1:M 15 Jun 2023 16:27:21.708 # Fatal: Can't initialize Background Jobs.
broker      | 1:C 15 Jun 2023 16:27:22.277 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
broker      | 1:C 15 Jun 2023 16:27:22.277 # Redis version=7.0.11, bits=64, commit=00000000, modified=0, pid=1, just started
broker      | 1:C 15 Jun 2023 16:27:22.277 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
broker      | 1:M 15 Jun 2023 16:27:22.278 * monotonic clock: POSIX clock_gettime
broker      | 1:M 15 Jun 2023 16:27:22.278 * Running mode=standalone, port=6379.
broker      | 1:M 15 Jun 2023 16:27:22.278 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
broker      | 1:M 15 Jun 2023 16:27:22.278 # Server initialized
broker      | 1:M 15 Jun 2023 16:27:22.278 # Fatal: Can't initialize Background Jobs.
broker      | 1:C 15 Jun 2023 16:27:22.946 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
broker      | 1:C 15 Jun 2023 16:27:22.946 # Redis version=7.0.11, bits=64, commit=00000000, modified=0, pid=1, just started
broker      | 1:C 15 Jun 2023 16:27:22.946 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
broker      | 1:M 15 Jun 2023 16:27:22.947 * monotonic clock: POSIX clock_gettime
broker      | 1:M 15 Jun 2023 16:27:22.947 * Running mode=standalone, port=6379.
broker      | 1:M 15 Jun 2023 16:27:22.947 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
broker      | 1:M 15 Jun 2023 16:27:22.947 # Server initialized
broker      | 1:M 15 Jun 2023 16:27:22.947 # Fatal: Can't initialize Background Jobs.
broker      | 1:C 15 Jun 2023 16:27:24.023 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
broker      | 1:C 15 Jun 2023 16:27:24.023 # Redis version=7.0.11, bits=64, commit=00000000, modified=0, pid=1, just started
broker      | 1:C 15 Jun 2023 16:27:24.023 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
broker      | 1:M 15 Jun 2023 16:27:24.023 * monotonic clock: POSIX clock_gettime
broker      | 1:M 15 Jun 2023 16:27:24.024 * Running mode=standalone, port=6379.
broker      | 1:M 15 Jun 2023 16:27:24.024 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
broker      | 1:M 15 Jun 2023 16:27:24.024 # Server initialized
broker      | 1:M 15 Jun 2023 16:27:24.024 # Fatal: Can't initialize Background Jobs.
broker      | 1:C 15 Jun 2023 16:27:25.895 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
broker      | 1:C 15 Jun 2023 16:27:25.895 # Redis version=7.0.11, bits=64, commit=00000000, modified=0, pid=1, just started
broker      | 1:C 15 Jun 2023 16:27:25.895 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf

Possible cause and solution

It looks like latest Redis docker images have been generated with a recent version of Docker, leading to incompatibilities with versions of docker shipped with Ubuntu Jammy and Lunar ( 20.10.08 for lunar ). See https://github.com/redis/redis/issues/12362#issuecomment-1617561165 for hints.

A workaround is to reinstall docker with latest APT packages from the official Docker repository : version 24 solves the issue and WebODM installation is ok and runs correctly.

This is still a problem though, as the WebODM installation method on any ubuntu box is currently broken by default.

A good practice would be to pin Docker images versions used in WebODM, so as not to be dependent on problems raised by new images, and update versions carefully whenever fully tested.

Thanks @vpicavet see if this is fixed with https://github.com/OpenDroneMap/WebODM/commit/f1b358db44b7db2e87e90d6c9f0f2f7ef4ea6b92 ? :pray:

It will probably fix future problems, but not this one I guess. Before upgrading docker, I tried to pin redis to latest 7 and 6 versions, without success. I am not sure they still have versions of containers available on the hub ,generated with docker < 20.10.10. Therefore, not sure pinning dependencies will matter this time.

We can :

• A/ work with upstream to see if they can fix the problem
• B/ update WebODM documentation to mention docker version > 20.10.10 is required

B seems the easiest and most straightforward option.

webodm.sh could be updated to check for the docker version.

I'm going to close this, as a workaround (update docker) is already available.