spiderfoot icon indicating copy to clipboard operation
spiderfoot copied to clipboard

Create outlier_web_server_header.yaml

Open geeknik opened this issue 2 years ago • 2 comments

Outlier detection -- A web server produced a header response which appeared in 10% or less of the total web servers found.

geeknik avatar Apr 12 '22 23:04 geeknik

Thanks for the submission! One question: how does this provide value over the outlier_webserver rule? I think this might be prone to be quite noisy since it's looking at the WEBSERVER_HTTPHEADERS type which can be very unique (e.g. due to timestamps, hashes, etc.) without actually telling you much about the target.

smicallef avatar Apr 19 '22 19:04 smicallef

Well I was hopeful that by using WEBSERVER_HTTPHEADERS it would only look at the header name, not the value. I'm more concerned about the name of the header vs the value of the header. But since I've never seen my correlation rule fire despite using it on quite a few scans with 150K or more elements, I wasn't sure what kind of data was going to be returned.

geeknik avatar Apr 19 '22 19:04 geeknik

I suspect this rule hasn't fired anything for you locally because the headers are too anomalous (because values are considered, not just the header, and even if we only looked at headers the headers are reported in aggregate, not individually), and the correlation engine has a safety mechanism for reporting outliers when the dataset is highly anomalous. I'd therefore suggest closing this PR until we implement some functionality to the engine that manipulates the data for processing.

smicallef avatar Sep 22 '22 11:09 smicallef