gvmd
gvmd copied to clipboard
Scans interrupted at high percentage
Expected behavior
GVM to complete the scan (reach 100% change its status from Running to Complete and Run the Alerts).
Actual behavior
GVM stuck at 99%. It has happened at 80%, and another few times at 95%+.
Steps to reproduce
This is hard as I couldn't always reproduce. At some Target lists, it completed flawlessly and in a fair amount of time. On other Targets, it can take up to 3 days stuck at 99% when it switches to Interrupted.
I was not sure what was causing it until find out recent logs and maybe this can help to solve it? On the last 4 scans that got stuck at 99% I saw the same error: ERROR: canceling statement due to user request
GVM versions
gsa: (gsad --version)
GSAD version 22.04.0
gvm: (gvmd --version)
Greenbone Vulnerability Manager version 22.4.0~dev1 (DB revision 250)
openvas-scanner: (openvas --version)
OSPd OpenVAS version 22.4.3.dev1.
GSAD version 22.04.0
gvm-libs:
latest at the moment of writing:
Digest:sha256:95f6eee25df254ede2a60ef0fdb8ed1e6a663835eaee16808c1bcd1a96ab7f87
Environment
Operating system:
Shouldn't matter but:
Distributor ID: Ubuntu
Description: Ubuntu 22.04.1 LTS
Release: 22.04
Codename: jammy
Installation method / source:
Greenbone GVM docker images with community docker-compose services.
Logfiles
gsa_1 | gsad gmp:WARNING:2022-11-25 16h38.13 GMT:13: Authentication failure for 'gvm' from 192.168.18.9. Status was 2.
gvmd_1 | md gmp:WARNING:2022-11-25 16h38.13 utc:6547: Authentication failure for 'gvm' from unix_socket
gsa_1 | gsad gmp:MESSAGE:2022-11-25 16h38.41 GMT:13: Authentication success for 'admin' from 192.168.18.9
gsa_1 | libgvm util:WARNING:2022-11-25 16h38.45 GMT:13: End error: Error on line 1 char 1: Document was empty or contained only whitespace
pg-gvm_1 | 2022-11-25 16:38:46.728 UTC [4157] gvmd@gvmd ERROR: canceling statement due to user request
pg-gvm_1 | 2022-11-25 16:38:46.728 UTC [4157] gvmd@gvmd CONTEXT: SQL function "report_severity_count" statement 1
pg-gvm_1 | 2022-11-25 16:38:46.728 UTC [4157] gvmd@gvmd STATEMENT: SELECT sum(aggregate_count) AS outer_count, aggregate_group_value AS outer_group_column, aggregate_subgroup_value AS outer_subgroup_column , max (text_column_0), max (text_column_1), max (text_column_2), max (text_column_3) FROM (WITH permissions_subject AS (SELECT * FROM permissions WHERE subject_location = 0 AND ((subject_type = 'user' AND subject = (SELECT id FROM users WHERE users.uuid = '933ce1ed-06f4-471e-aa20-debfd2bd76d6')) OR (subject_type = 'group' AND subject IN (SELECT DISTINCT "group" FROM group_users WHERE "user" = (SELECT id FROM users WHERE users.uuid = '933ce1ed-06f4-471e-aa20-debfd2bd76d6'))) OR (subject_type = 'role' AND subject IN (SELECT DISTINCT role FROM role_users WHERE "user" = (SELECT id FROM users WHERE users.uuid = '933ce1ed-06f4-471e-aa20-debfd2bd76d6'))))), super_on_users AS (SELECT DISTINCT * FROM (SELECT resource FROM permissions_subject WHERE name = 'Super' AND resource_type = 'user' UNION SELECT "user" FROM role_users WHERE role IN (SELECT resource FROM permissions_subject WHERE name = 'Super' AND resource_type = 'role') UNION SELECT "user" FROM group_users WHERE "group" IN (SELECT resource FROM permissions_subject WHERE name = 'Super' AND resource_type = 'group')) AS all_users)SELECT count(*) AS aggregate_count, uuid AS aggregate_group_value, CAST (NULL AS TEXT) AS aggregate_subgroup_value, max (CASE WHEN target IS null OR opts.ignore_severity != 0 THEN 0 ELSE coalesce (report_severity_count (task_last_report (id), opts.override, opts.min_qod, 'High') * 1.0 / nullif (report_result_host_count (task_last_report (id), opts.min_qod), 0), 0) END) as text_column_0, max (modification_time) as text_column_1, max (name) as text_column_2, max (task_severity (id, opts.override, opts.min_qod)) as text_column_3 FROM tasks, (SELECT 0 AS override, 70 AS min_qod, 0 AS ignore_severity) AS opts WHERE ((tasks.owner = (SELECT id FROM users WHERE users.uuid = '933ce1ed-06f4-471e-aa20-debfd2bd76d6')) OR EXISTS (SELECT * FROM permissions_subject WHERE name = 'Super' AND (resource = 0)) OR tasks.owner IN (SELECT * FROM super_on_users) OR EXISTS (SELECT id FROM permissions_subject WHERE resource = tasks.id AND resource_type = 'task' AND resource_location = 0 AND (t ()))) AND hidden = 0 AND usage_type = 'scan' GROUP BY uuid ) AS agg_sub GROUP BY outer_group_column, outer_subgroup_column ORDER BY outer_group_column DESC, max (text_column_0) DESC LIMIT 10 OFFSET 0;
gsa_1 | libgvm util:WARNING:2022-11-25 16h38.45 GMT:13: End error: Error on line 1 char 1: Document was empty or contained only whitespace
pg-gvm_1 | 2022-11-25 16:38:46.956 UTC [4155] gvmd@gvmd ERROR: canceling statement due to user request
pg-gvm_1 | 2022-11-25 16:38:46.956 UTC [4155] gvmd@gvmd CONTEXT: SQL function "report_severity_count" statement 1
pg-gvm_1 | 2022-11-25 16:38:46.956 UTC [4155] gvmd@gvmd STATEMENT: SELECT sum(aggregate_count) AS outer_count, aggregate_group_value AS outer_group_column, aggregate_subgroup_value AS outer_subgroup_column , max (text_column_0), max (text_column_1), max (text_column_2), max (text_column_3) FROM (WITH permissions_subject AS (SELECT * FROM permissions WHERE subject_location = 0 AND ((subject_type = 'user' AND subject = (SELECT id FROM users WHERE users.uuid = '933ce1ed-06f4-471e-aa20-debfd2bd76d6')) OR (subject_type = 'group' AND subject IN (SELECT DISTINCT "group" FROM group_users WHERE "user" = (SELECT id FROM users WHERE users.uuid = '933ce1ed-06f4-471e-aa20-debfd2bd76d6'))) OR (subject_type = 'role' AND subject IN (SELECT DISTINCT role FROM role_users WHERE "user" = (SELECT id FROM users WHERE users.uuid = '933ce1ed-06f4-471e-aa20-debfd2bd76d6'))))), super_on_users AS (SELECT DISTINCT * FROM (SELECT resource FROM permissions_subject WHERE name = 'Super' AND resource_type = 'user' UNION SELECT "user" FROM role_users WHERE role IN (SELECT resource FROM permissions_subject WHERE name = 'Super' AND resource_type = 'role') UNION SELECT "user" FROM group_users WHERE "group" IN (SELECT resource FROM permissions_subject WHERE name = 'Super' AND resource_type = 'group')) AS all_users)SELECT count(*) AS aggregate_count, uuid AS aggregate_group_value, CAST (NULL AS TEXT) AS aggregate_subgroup_value, max (CASE WHEN target IS null OR opts.ignore_severity != 0 THEN 0 ELSE coalesce (report_severity_count (task_last_report (id), opts.override, opts.min_qod, 'High') * 1.0 / nullif (report_result_host_count (task_last_report (id), opts.min_qod), 0), 0) END) as text_column_0, max (modification_time) as text_column_1, max (name) as text_column_2, max (task_severity (id, opts.override, opts.min_qod)) as text_column_3 FROM tasks, (SELECT 0 AS override, 70 AS min_qod, 0 AS ignore_severity) AS opts WHERE ((tasks.owner = (SELECT id FROM users WHERE users.uuid = '933ce1ed-06f4-471e-aa20-debfd2bd76d6')) OR EXISTS (SELECT * FROM permissions_subject WHERE name = 'Super' AND (resource = 0)) OR tasks.owner IN (SELECT * FROM super_on_users) OR EXISTS (SELECT id FROM permissions_subject WHERE resource = tasks.id AND resource_type = 'task' AND resource_location = 0 AND (t ()))) AND hidden = 0 AND usage_type = 'scan' GROUP BY uuid ) AS agg_sub GROUP BY outer_group_column, outer_subgroup_column ORDER BY outer_group_column DESC, max (text_column_0) DESC LIMIT 10 OFFSET 0;
gsa_1 | gsad gmp:MESSAGE:2022-11-25 18h13.35 GMT:13: Authentication success for 'admin' from 192.168.18.9
gsa_1 | libgvm util:WARNING:2022-11-25 18h14.19 GMT:13: End error: Error on line 1 char 1: Document was empty or contained only whitespace
gvmd_1 | md main:WARNING:2022-11-25 18h14.20 UTC:7270: write_to_client_unix: failed to write to client: Broken pipe
mqtt-broker_1 | 1669478209: Client 3576ba88-02c9-4585-b236-45922a0a19c3 has exceeded timeout, disconnecting.
gsa_1 | gsad gmp:MESSAGE:2022-11-27 22h43.10 GMT:13: Authentication success for 'admin' from 192.168.18.9
gvmd_1 | event task:MESSAGE:2022-11-27 22h45.52 UTC:26438: Status of task vip-staging internal hosts (1b97362e-9a49-48cc-b29c-2dd8e81aa468) has changed to Requested
gvmd_1 | event task:MESSAGE:2022-11-27 22h45.52 UTC:26438: Task vip-staging internal hosts (1b97362e-9a49-48cc-b29c-2dd8e81aa468) has been requested to start by admin
gsa_1 | libgvm util:WARNING:2022-11-27 22h45.54 GMT:13: End error: Error on line 1 char 1: Document was empty or contained only whitespace
pg-gvm_1 | 2022-11-27 22:45:55.379 UTC [30276] gvmd@gvmd ERROR: canceling statement due to user request
pg-gvm_1 | 2022-11-27 22:45:55.379 UTC [30276] gvmd@gvmd CONTEXT: SQL function "report_severity_count" statement 1
pg-gvm_1 | 2022-11-27 22:45:55.379 UTC [30276] gvmd@gvmd STATEMENT: SELECT sum(aggregate_count) AS outer_count, aggregate_group_value AS outer_group_column, aggregate_subgroup_value AS outer_subgroup_column , max (text_column_0), max (text_column_1), max (text_column_2), max (text_column_3) FROM (WITH permissions_subject AS (SELECT * FROM permissions WHERE subject_location = 0 AND ((subject_type = 'user' AND subject = (SELECT id FROM users WHERE users.uuid = '933ce1ed-06f4-471e-aa20-debfd2bd76d6')) OR (subject_type = 'group' AND subject IN (SELECT DISTINCT "group" FROM group_users WHERE "user" = (SELECT id FROM users WHERE users.uuid = '933ce1ed-06f4-471e-aa20-debfd2bd76d6'))) OR (subject_type = 'role' AND subject IN (SELECT DISTINCT role FROM role_users WHERE "user" = (SELECT id FROM users WHERE users.uuid = '933ce1ed-06f4-471e-aa20-debfd2bd76d6'))))), super_on_users AS (SELECT DISTINCT * FROM (SELECT resource FROM permissions_subject WHERE name = 'Super' AND resource_type = 'user' UNION SELECT"user" FROM role_users WHERE role IN (SELECT resource FROM permissions_subject WHERE name = 'Super' AND resource_type = 'role') UNION SELECT "user" FROM group_users WHERE "group" IN (SELECT resource FROM permissions_subject WHERE name = 'Super' AND resource_type = 'group')) AS all_users)SELECT count(*) AS aggregate_count, uuid AS aggregate_group_value, CAST (NULL AS TEXT) AS aggregate_subgroup_value, max (CASE WHEN target IS null OR opts.ignore_severity != 0 THEN 0 ELSE coalesce (report_severity_count (task_last_report (id), opts.override, opts.min_qod, 'High') * 1.0 / nullif (report_result_host_count (task_last_report (id), opts.min_qod), 0), 0) END) as text_column_0, max (modification_time) as text_column_1, max (name) as text_column_2, max (task_severity (id, opts.override, opts.min_qod)) as text_column_3 FROM tasks, (SELECT 0 AS override, 70 AS min_qod, 0 AS ignore_severity) AS opts WHERE ((tasks.owner = (SELECT id FROM users WHERE users.uuid = '933ce1ed-06f4-471e-aa20-debfd2bd76d6')) OR EXISTS (SELECT * FROM permissions_subject WHERE name = 'Super' AND (resource = 0)) OR tasks.owner IN (SELECT * FROM super_on_users) OR EXISTS (SELECT id FROM permissions_subject WHERE resource = tasks.id AND resource_type = 'task' AND resource_location = 0 AND (t ()))) AND hidden = 0 AND usage_type = 'scan' GROUP BY uuid ) AS agg_sub GROUP BY outer_group_column, outer_subgroup_column ORDER BY outer_group_column DESC, max (text_column_0) DESC LIMIT 10 OFFSET 0;