purldb
purldb copied to clipboard
nexB
docker compose exec web purldb create-scan-queue-worker-user scan_worker_1 fails
I'm trying to follow https://aboutcode.readthedocs.io/projects/PURLdb/en/latest/how-to-guides/installation.html#purldb
Ignoring the second "make envfile" I'm getting as far as docker compose up:
[+] Running 7/7
✔ Container purldb-redis-1 Created 0.2s
✔ Container purldb-db-1 Created 0.2s
✔ Container traefik Created 0.2s
✔ Container purldb-web-1 Created 0.0s
✔ Container purldb-rq_worker-1 Created 0.1s
✔ Container purldb-nginx-1 Created 0.1s
✔ Container purldb-scheduler-1 Created 0.1s
Attaching to db-1, nginx-1, redis-1, rq_worker-1, scheduler-1, web-1, traefik
redis-1 | Starting Redis Server
redis-1 | 1:C 11 Jun 2025 16:05:02.850 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis-1 | 1:C 11 Jun 2025 16:05:02.851 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis-1 | 1:C 11 Jun 2025 16:05:02.851 * Redis version=8.0.2, bits=64, commit=00000000, modified=1, pid=1, just started
redis-1 | 1:C 11 Jun 2025 16:05:02.851 * Configuration loaded
redis-1 | 1:M 11 Jun 2025 16:05:02.854 * monotonic clock: POSIX clock_gettime
redis-1 | 1:M 11 Jun 2025 16:05:02.858 * Running mode=standalone, port=6379.
redis-1 | 1:M 11 Jun 2025 16:05:02.864 * <bf> RedisBloom version 8.0.1 (Git=unknown)
redis-1 | 1:M 11 Jun 2025 16:05:02.864 * <bf> Registering configuration options: [
redis-1 | 1:M 11 Jun 2025 16:05:02.864 * <bf> { bf-error-rate : 0.01 }
redis-1 | 1:M 11 Jun 2025 16:05:02.864 * <bf> { bf-initial-size : 100 }
redis-1 | 1:M 11 Jun 2025 16:05:02.864 * <bf> { bf-expansion-factor : 2 }
redis-1 | 1:M 11 Jun 2025 16:05:02.864 * <bf> { cf-bucket-size : 2 }
redis-1 | 1:M 11 Jun 2025 16:05:02.864 * <bf> { cf-initial-size : 1024 }
redis-1 | 1:M 11 Jun 2025 16:05:02.864 * <bf> { cf-max-iterations : 20 }
redis-1 | 1:M 11 Jun 2025 16:05:02.864 * <bf> { cf-expansion-factor : 1 }
redis-1 | 1:M 11 Jun 2025 16:05:02.864 * <bf> { cf-max-expansions : 32 }
redis-1 | 1:M 11 Jun 2025 16:05:02.864 * <bf> ]
redis-1 | 1:M 11 Jun 2025 16:05:02.865 * Module 'bf' loaded from /usr/local/lib/redis/modules//redisbloom.so
redis-1 | 1:M 11 Jun 2025 16:05:02.894 * <search> Redis version found by RedisSearch : 8.0.2 - oss
redis-1 | 1:M 11 Jun 2025 16:05:02.894 * <search> RediSearch version 8.0.1 (Git=5688fcc)
redis-1 | 1:M 11 Jun 2025 16:05:02.894 * <search> Low level api version 1 initialized successfully
redis-1 | 1:M 11 Jun 2025 16:05:02.896 * <search> gc: ON, prefix min length: 2, min word length to stem: 4, prefix max expansions: 200, query timeout (ms): 500, timeout policy: return, cursor read size: 1000, cursor max idle (ms): 300000, max doctable size: 1000000, max number of search results: 1000000,
redis-1 | 1:M 11 Jun 2025 16:05:02.897 * <search> Initialized thread pools!
redis-1 | 1:M 11 Jun 2025 16:05:02.897 * <search> Disabled workers threadpool of size 0
redis-1 | 1:M 11 Jun 2025 16:05:02.898 * <search> Subscribe to config changes
redis-1 | 1:M 11 Jun 2025 16:05:02.898 * <search> Enabled role change notification
redis-1 | 1:M 11 Jun 2025 16:05:02.901 * <search> Cluster configuration: AUTO partitions, type: 0, coordinator timeout: 0ms
redis-1 | 1:M 11 Jun 2025 16:05:02.902 * <search> Register write commands
redis-1 | 1:M 11 Jun 2025 16:05:02.903 * Module 'search' loaded from /usr/local/lib/redis/modules//redisearch.so
redis-1 | 1:M 11 Jun 2025 16:05:02.909 * <timeseries> RedisTimeSeries version 80001, git_sha=577bfa8b5909e7ee572f0b651399be8303dc6641
redis-1 | 1:M 11 Jun 2025 16:05:02.909 * <timeseries> Redis version found by RedisTimeSeries : 8.0.2 - oss
redis-1 | 1:M 11 Jun 2025 16:05:02.909 * <timeseries> Registering configuration options: [
redis-1 | 1:M 11 Jun 2025 16:05:02.909 * <timeseries> { ts-compaction-policy : }
redis-1 | 1:M 11 Jun 2025 16:05:02.909 * <timeseries> { ts-num-threads : 3 }
redis-1 | 1:M 11 Jun 2025 16:05:02.909 * <timeseries> { ts-retention-policy : 0 }
redis-1 | 1:M 11 Jun 2025 16:05:02.909 * <timeseries> { ts-duplicate-policy : block }
redis-1 | 1:M 11 Jun 2025 16:05:02.909 * <timeseries> { ts-chunk-size-bytes : 4096 }
redis-1 | 1:M 11 Jun 2025 16:05:02.909 * <timeseries> { ts-encoding : compressed }
redis-1 | 1:M 11 Jun 2025 16:05:02.909 * <timeseries> { ts-ignore-max-time-diff: 0 }
redis-1 | 1:M 11 Jun 2025 16:05:02.909 * <timeseries> { ts-ignore-max-val-diff : 0.000000 }
redis-1 | 1:M 11 Jun 2025 16:05:02.909 * <timeseries> ]
redis-1 | 1:M 11 Jun 2025 16:05:02.910 * <timeseries> Detected redis oss
redis-1 | 1:M 11 Jun 2025 16:05:02.911 * Module 'timeseries' loaded from /usr/local/lib/redis/modules//redistimeseries.so
redis-1 | 1:M 11 Jun 2025 16:05:02.924 * <ReJSON> Created new data type 'ReJSON-RL'
redis-1 | 1:M 11 Jun 2025 16:05:02.927 * <ReJSON> version: 80001 git sha: unknown branch: unknown
redis-1 | 1:M 11 Jun 2025 16:05:02.927 * <ReJSON> Exported RedisJSON_V1 API
redis-1 | 1:M 11 Jun 2025 16:05:02.927 * <ReJSON> Exported RedisJSON_V2 API
redis-1 | 1:M 11 Jun 2025 16:05:02.927 * <ReJSON> Exported RedisJSON_V3 API
redis-1 | 1:M 11 Jun 2025 16:05:02.927 * <ReJSON> Exported RedisJSON_V4 API
redis-1 | 1:M 11 Jun 2025 16:05:02.927 * <ReJSON> Exported RedisJSON_V5 API
redis-1 | 1:M 11 Jun 2025 16:05:02.927 * <ReJSON> Enabled diskless replication
redis-1 | 1:M 11 Jun 2025 16:05:02.929 * <ReJSON> Initialized shared string cache, thread safe: false.
redis-1 | 1:M 11 Jun 2025 16:05:02.929 * Module 'ReJSON' loaded from /usr/local/lib/redis/modules//rejson.so
redis-1 | 1:M 11 Jun 2025 16:05:02.929 * <search> Acquired RedisJSON_V5 API
redis-1 | 1:M 11 Jun 2025 16:05:02.930 * Server initialized
redis-1 | 1:M 11 Jun 2025 16:05:02.930 * <search> Loading event starts
redis-1 | 1:M 11 Jun 2025 16:05:02.931 * <search> Enabled workers threadpool of size 4
redis-1 | 1:M 11 Jun 2025 16:05:02.931 * Reading RDB base file on AOF loading...
redis-1 | 1:M 11 Jun 2025 16:05:02.932 * Loading RDB produced by version 8.0.2
redis-1 | 1:M 11 Jun 2025 16:05:02.932 * RDB age 87916 seconds
redis-1 | 1:M 11 Jun 2025 16:05:02.932 * RDB memory usage when created 0.92 Mb
redis-1 | 1:M 11 Jun 2025 16:05:02.932 * RDB is base AOF
redis-1 | 1:M 11 Jun 2025 16:05:02.932 * Done loading RDB, keys loaded: 0, keys expired: 0.
redis-1 | 1:M 11 Jun 2025 16:05:02.934 * DB loaded from base file appendonly.aof.1.base.rdb: 0.004 seconds
redis-1 | 1:M 11 Jun 2025 16:05:02.937 * DB loaded from incr file appendonly.aof.1.incr.aof: 0.002 seconds
redis-1 | 1:M 11 Jun 2025 16:05:02.937 * <search> Disabled workers threadpool of size 4
redis-1 | 1:M 11 Jun 2025 16:05:02.937 * <search> Loading event ends
redis-1 | 1:M 11 Jun 2025 16:05:02.937 * DB loaded from append only file: 0.006 seconds
redis-1 | 1:M 11 Jun 2025 16:05:02.944 * Opening AOF incr file appendonly.aof.1.incr.aof on server start
redis-1 | 1:M 11 Jun 2025 16:05:02.944 * Ready to accept connections tcp
db-1 |
db-1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
db-1 |
db-1 | 2025-06-11 16:05:03.209 UTC [1] LOG: starting PostgreSQL 13.21 (Debian 13.21-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
db-1 | 2025-06-11 16:05:03.210 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db-1 | 2025-06-11 16:05:03.210 UTC [1] LOG: listening on IPv6 address "::", port 5432
db-1 | 2025-06-11 16:05:03.215 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db-1 | 2025-06-11 16:05:03.228 UTC [27] LOG: database system was shut down at 2025-06-10 15:45:52 UTC
db-1 | 2025-06-11 16:05:03.270 UTC [1] LOG: database system is ready to accept connections
web-1 | wait-for-it: waiting 60 seconds for db:5432
db-1 | 2025-06-11 16:05:03.330 UTC [34] LOG: incomplete startup packet
web-1 | wait-for-it: db:5432 is available after 0 seconds
traefik | time="2025-06-11T16:05:03Z" level=info msg="Configuration loaded from file: /traefik.yml"
traefik | time="2025-06-11T16:05:03Z" level=info msg="Traefik version 2.10.7 built on 2023-12-06T15:54:59Z"
traefik | time="2025-06-11T16:05:03Z" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://doc.traefik.io/traefik/contributing/data-collection/\n"
traefik | time="2025-06-11T16:05:03Z" level=info msg="Starting provider aggregator aggregator.ProviderAggregator"
traefik | time="2025-06-11T16:05:03Z" level=info msg="Starting provider *traefik.Provider"
traefik | time="2025-06-11T16:05:03Z" level=info msg="Starting provider *docker.Provider"
traefik | time="2025-06-11T16:05:03Z" level=info msg="Starting provider *acme.ChallengeTLSALPN"
nginx-1 | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
nginx-1 | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
nginx-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
nginx-1 | 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
nginx-1 | 10-listen-on-ipv6-by-default.sh: info: /etc/nginx/conf.d/default.conf differs from the packaged version
nginx-1 | /docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
nginx-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
nginx-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
nginx-1 | /docker-entrypoint.sh: Configuration complete; ready for start up
nginx-1 | 2025/06/11 16:05:03 [notice] 1#1: using the "epoll" event method
nginx-1 | 2025/06/11 16:05:03 [notice] 1#1: nginx/1.27.5
nginx-1 | 2025/06/11 16:05:03 [notice] 1#1: built by gcc 12.2.0 (Debian 12.2.0-14)
nginx-1 | 2025/06/11 16:05:03 [notice] 1#1: OS: Linux 6.1.0-37-amd64
nginx-1 | 2025/06/11 16:05:03 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
nginx-1 | 2025/06/11 16:05:03 [notice] 1#1: start worker processes
nginx-1 | 2025/06/11 16:05:03 [notice] 1#1: start worker process 28
nginx-1 | 2025/06/11 16:05:03 [notice] 1#1: start worker process 29
nginx-1 | 2025/06/11 16:05:03 [notice] 1#1: start worker process 30
nginx-1 | 2025/06/11 16:05:03 [notice] 1#1: start worker process 31
nginx-1 | 2025/06/11 16:05:03 [notice] 1#1: start worker process 32
nginx-1 | 2025/06/11 16:05:03 [notice] 1#1: start worker process 33
nginx-1 | 2025/06/11 16:05:03 [notice] 1#1: start worker process 34
nginx-1 | 2025/06/11 16:05:03 [notice] 1#1: start worker process 35
rq_worker-1 | wait-for-it: waiting 15 seconds for web:8000
scheduler-1 | wait-for-it: waiting 15 seconds for web:8000
web-1 | Operations to perform:
web-1 | Apply all migrations: admin, auth, authtoken, clearcode, contenttypes, django_rq, matchcode, minecode, packagedb, sessions
web-1 | Running migrations:
web-1 | No migrations to apply.
web-1 | [2025-06-11 16:05:12 +0000] [20] [INFO] Starting gunicorn 23.0.0
web-1 | [2025-06-11 16:05:12 +0000] [20] [INFO] Listening at: http://0.0.0.0:8000 (20)
web-1 | [2025-06-11 16:05:12 +0000] [20] [INFO] Using worker: sync
web-1 | [2025-06-11 16:05:12 +0000] [21] [INFO] Booting worker with pid: 21
web-1 | [2025-06-11 16:05:12 +0000] [22] [INFO] Booting worker with pid: 22
web-1 | [2025-06-11 16:05:12 +0000] [23] [INFO] Booting worker with pid: 23
web-1 | [2025-06-11 16:05:12 +0000] [24] [INFO] Booting worker with pid: 24
web-1 | [2025-06-11 16:05:12 +0000] [25] [INFO] Booting worker with pid: 25
web-1 | [2025-06-11 16:05:12 +0000] [26] [INFO] Booting worker with pid: 26
web-1 | [2025-06-11 16:05:12 +0000] [27] [INFO] Booting worker with pid: 27
web-1 | [2025-06-11 16:05:12 +0000] [28] [INFO] Booting worker with pid: 28
rq_worker-1 | wait-for-it: web:8000 is available after 9 seconds
scheduler-1 | wait-for-it: web:8000 is available after 9 seconds
rq_worker-1 | INFO:rq.worker:Worker rq:worker:c5f1268cb6a649b7b809490b67189275 started with PID 1, version 2.1.0
rq_worker-1 | INFO:rq.worker:Subscribing to channel rq:pubsub:c5f1268cb6a649b7b809490b67189275
rq_worker-1 | INFO:rq.worker:*** Listening on default...
rq_worker-1 | INFO:rq.worker:Cleaning registries for queue: default
scheduler-1 | 16:05:18 Registering birth
scheduler-1 | INFO:rq_scheduler.scheduler:Registering birth
But then docker compose exec web purldb create-scan-queue-worker-user scan_worker_1 throws an error:
Traceback (most recent call last):
File "/opt/purldb/venv/bin/purldb", line 5, in <module>
from purldb import command_line
ModuleNotFoundError: No module named 'purldb'
