HTTP collector: Log response size

Open ghost opened this issue 4 years ago • 1 comments

Proposal for the HTTP collector log. Instead of

INFO - Downloading report from 'https://urlhaus.abuse.ch/feeds/tld/at/'.

log the size in parenthesis:

INFO - Downloading report from 'https://urlhaus.abuse.ch/feeds/tld/at/' (1.4 KB).

Then it's easy to see if a response was empty (0B, see also https://lists.cert.at/pipermail/intelmq-users/2021-February/000202.html) or very big.

Mar 01 '21 10:03 ghost

len(response) will give the size in bytes (after unpacking). We can add a function to intelmq.lib.utils for size-conversion to get a human-readable size specifier.

Mar 15 '21 08:03 ghost