Option to adjust SimpleQueue logging?

[KengTeong]

Greetings! I have a question on the chart for passing extra vars to limit the amount of noise generated from the SimpleQueue service.

Anchore Version Information: Anchore Engine (preloaded-db) v0.10.1

Below is an example where I attempted to pass an extra variable for logging telling it to only post debug level logs. (Note* it is now commented out since it did not work)

However, when I checked the logs in Argo it is still spamming the log. It appears as though the log is actually generated from the python requests module.

Questions:

1. Is there a way I can minimize the amount of logging information from the SimpleQueue service? Either by an extra variable or other option?

2. What Arguments can be passed to SimpleQueue

3. Can we change the polling interval

4. Is 0 or -1 off for values

[Btodhunter]

@KengTeong Thanks for the input! Currently the chart does not have any mechanisms for adjusting the log level for individual services. The only option at the moment is Values.anchoreGlobal.logLevel which sets the log level for all services.

This would be a good addition to the chart though, so I'm leaving this issue open.

[frumpleswift]

I work with @KengTeong, and tested changing Values.anchoreGlobal.logLevel.

I set the logLevel to ERROR and redeployed anchore. However, the simple queue logs were not affected by the change.

[user@bastion ~]$ k logs -n anchore anchore-engine-anchore-engine-simplequeue-6d4f7464bd-lvvnv | tail -n 20 [service:simplequeue] 2021-10-22 22:11:01+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:01 +0000] "GET /v1/leases/analyzer_queue HTTP/1.1" 200 87 "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:01+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:01 +0000] "GET /v1/leases/analyzer_queue/acquire?client_id=anchore-engine-anchore-engine-catalog-5994fdf7c-brts2%3A15%3A139708675897088%3A&ttl=60 HTTP/1.1" 200 182 "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:01+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:01 +0000] "GET /v1/leases/analyzer_queue/release?client_id=anchore-engine-anchore-engine-catalog-5994fdf7c-brts2%3A15%3A139708675897088%3A&epoch=1745 HTTP/1.1" 200 - "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:03+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:03 +0000] "POST /v1/queues/watcher_tasks/is_inqueue HTTP/1.1" 200 3 "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:03+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:03 +0000] "POST /v1/queues/watcher_tasks?qcount=0&forcefirst=False HTTP/1.1" 200 5 "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:03+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:03 +0000] "GET /v1/queues/watcher_tasks?wait_max_seconds=30&visibility_timeout=0 HTTP/1.1" 200 451 "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:03+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:03 +0000] "GET /v1/leases/analyzer_queue HTTP/1.1" 200 87 "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:03+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:03 +0000] "GET /v1/leases/analyzer_queue/acquire?client_id=anchore-engine-anchore-engine-catalog-5994fdf7c-brts2%3A15%3A139708692682496%3A&ttl=60 HTTP/1.1" 200 182 "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:04+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:03 +0000] "GET /v1/leases/analyzer_queue/release?client_id=anchore-engine-anchore-engine-catalog-5994fdf7c-brts2%3A15%3A139708692682496%3A&epoch=1747 HTTP/1.1" 200 - "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:05+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:05 +0000] "POST /v1/queues/watcher_tasks/is_inqueue HTTP/1.1" 200 3 "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:05+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:05 +0000] "POST /v1/queues/watcher_tasks?qcount=0&forcefirst=False HTTP/1.1" 200 5 "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:05+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:05 +0000] "POST /v1/queues/watcher_tasks/is_inqueue HTTP/1.1" 200 3 "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:05+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:05 +0000] "POST /v1/queues/watcher_tasks?qcount=0&forcefirst=False HTTP/1.1" 200 5 "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:05+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:05 +0000] "POST /v1/queues/watcher_tasks/is_inqueue HTTP/1.1" 200 3 "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:05+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:05 +0000] "POST /v1/queues/watcher_tasks?qcount=0&forcefirst=False HTTP/1.1" 200 5 "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:06+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:05 +0000] "GET /v1/queues/watcher_tasks?wait_max_seconds=30&visibility_timeout=0 HTTP/1.1" 200 451 "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:06+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:05 +0000] "GET /v1/leases/analyzer_queue HTTP/1.1" 200 87 "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:06+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:05 +0000] "GET /v1/queues/watcher_tasks?wait_max_seconds=30&visibility_timeout=0 HTTP/1.1" 200 449 "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:06+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:05 +0000] "GET /v1/leases/analyzer_queue/acquire?client_id=anchore-engine-anchore-engine-catalog-5994fdf7c-brts2%3A15%3A139708659111680%3A&ttl=60 HTTP/1.1" 200 182 "-" "python-requests/2.26.0" [service:simplequeue] 2021-10-22 22:11:06+0000 [-] "127.0.0.1" - - [22/Oct/2021:22:11:05 +0000] "GET /v1/leases/notifications HTTP/1.1" 200 85 "-" "python-requests/2.26.0" [user@bastion ~]$

[Btodhunter]

@frumpleswift Thanks for the follow up. Can you do me a favor and shell into the simplequeue service and head /config/config.yaml an verify that the config has log_level: ERROR in it?

[frumpleswift]

Here is the config file:

[user@bastion ~]$ k exec -it -n anchore anchore-engine-anchore-engine-simplequeue-6d4f7464bd-lvvnv -- head /config/config.yaml
Defaulting container name to anchore-engine-simplequeue.
Use kubectl describe pod/anchore-engine-anchore-engine-simplequeue-6d4f7464bd-lvvnv -n anchore to see all of the containers in this pod.
# Anchore Service Configuration File from ConfigMap
service_dir: /anchore_service
tmp_dir: /analysis_scratch
log_level: ERROR
image_analyze_timeout_seconds: 36000
allow_awsecr_iam_auto: false
host_id: "${ANCHORE_POD_NAME}"
internal_ssl_verify: false
auto_restart_services: false
[user@bastion ~]$ 

and just to show logs once more:

[user@bastion ~]$ k logs -n anchore anchore-engine-anchore-engine-simplequeue-6d4f7464bd-lvvnv | tail -n 5
[service:simplequeue] 2021-10-23 01:25:21+0000 [-] "127.0.0.1" - - [23/Oct/2021:01:25:21 +0000] "POST /v1/queues/watcher_tasks/is_inqueue HTTP/1.1" 200 3 "-" "python-requests/2.26.0"
[service:simplequeue] 2021-10-23 01:25:21+0000 [-] "127.0.0.1" - - [23/Oct/2021:01:25:21 +0000] "POST /v1/queues/watcher_tasks?qcount=0&forcefirst=False HTTP/1.1" 200 5 "-" "python-requests/2.26.0"
[service:simplequeue] 2021-10-23 01:25:21+0000 [-] "127.0.0.1" - - [23/Oct/2021:01:25:21 +0000] "GET /v1/queues/watcher_tasks?wait_max_seconds=30&visibility_timeout=0 HTTP/1.1" 200 451 "-" "python-requests/2.26.0"
[service:simplequeue] 2021-10-23 01:25:21+0000 [-] "127.0.0.1" - - [23/Oct/2021:01:25:21 +0000] "GET /v1/leases/analyzer_queue HTTP/1.1" 200 88 "-" "python-requests/2.26.0"
[service:simplequeue] 2021-10-23 01:25:21+0000 [-] "127.0.0.1" - - [23/Oct/2021:01:25:21 +0000] "GET /v1/leases/analyzer_queue/acquire?client_id=anchore-engine-anchore-engine-catalog-5994fdf7c-brts2%3A15%3A139708130633472%3A&ttl=60 HTTP/1.1" 200 183 "-" "python-requests/2.26.0"
[user@bastion ~]$ 

[frumpleswift]

@Btodhunter do you need any additional information?