run testssl.sh as Icinga / Nagios Plugin, exit codes

[jenslink]

I think it would be great to run testssl.sh as Icinga / Nagios plugin.

See http://docs.icinga.org/latest/de/pluginapi.html for details about plugin development.

[drwetter]

I think also a proper return value would be fine, right?

[jenslink]

Yes. Return value and Text are important. Without setting the return value to something different then 0 the check will always be okay and without text you'll have to figure out why the check is not okay on your own.

[drwetter]

Jens,

thx,

Could you specify what text is needed. I supposed only the finding (if any), right?

Cheers, Dirk

[jenslink]

showing only the warnings and critical findings should be enough.

[matteocorti]

From the specification is just text but usually

• the text begins with the plugin name (all capitals)
• the status (all capitals)
• some summary

Something like

TESTSSL CRITICAL SSLv2 offered

As the list of problems could be long I would rather just say something like "3 issues detected" and then the details in the next lines (the additional lines are not shown in the issues summary)

TESTSSL CRITICAL 2 issues
SSLv3 offered
SSLv2 offered

[sni]

Nagios plugins require a summary output as first line of the output, followed by the details. So you would have to save the output and print only the results at the end. The complete specification for nagios plugins can be found here: https://www.monitoring-plugins.org/doc/guidelines.html

[AlGreed]

I will take it over. With the recently added severity levels it should work well. Icinga has UNKNOWN, OK, WARNING, CRITICAL states for any check. testssl will send a state (exitcode) and a report, something like "STATE:CRITICAL [LOW:1, MEDIUM:0, HIGH:5, CRITICAL:1]"

[drwetter]

Very good!

Sent from my mobile. Excuse my brevity&typos+the phone's autocorrection

[AlGreed]

Question to our bash pros:

original code to run checks:

$do_heartbleed && { run_heartbleed; ret=$(($? + ret)); }
$do_ccs_injection && { run_ccs_injection; ret=$(($? + ret)); }
$do_renego && { run_renego; ret=$(($? + ret)); }
$do_crime && { run_crime; ret=$(($? + ret)); }

I have problem to get return value in a new construction, not doing it to complex:

$do_ccs_injection && VULN_COUNTER=$(combine_vulnerability_reports "$VULN_COUNTER" "$(run_ccs_injection)")	 
$do_renego && VULN_COUNTER=$(combine_vulnerability_reports "$VULN_COUNTER" "$(run_renego)")
$do_crime && VULN_COUNTER=$(combine_vulnerability_reports "$VULN_COUNTER" "$(run_crime)")

Is it possible somehow to set return in this construction or i should use if ... then ... ?:

if [[ $do_heartbleed ]]; then
        report="$(run_heartbleed)"
        ret=$(($? + ret))
        VULN_COUNTER=$(combine_vulnerability_reports "$VULN_COUNTER" "$report") 
fi

[drwetter]

From the architectural point I would say that we first decide whether this is a good approach.

Historically the return values had the meaning of a successful completion of the run function. That changed but not consequently.

Without looking at the current code maybe it's a good idea to reconsider whether we like the original idea of successful completion and pass any result of a run_ function rather to global variable.

I also would do a similar thing for a rating.

Let me look into it later.

-- Sent from my mobile. Excuse my brevity&typos+the phone's autocorrection

[AlGreed]

From the architectural point I would say that we first decide whether this is a good approach.

It has turned out in a not simple task, i would said. It seems it will be a big changes in code to adapt testssl for icinga. So i decided to try some variants firstly and then discuss them.

to global variable.

My mistake. At first it was global, but now local. I forgot to change it back to low case.

I am experimenting with vulnerabilities section at the moment. My first approach is the following:

1. I needed some new args:

CONSOLE_MODE=true             # Console output
NAGIOS=false                  # Prepare vulnerability assessment report for Nagios

I needed a way to turn off the console output, so i decided to add CONSOLE_MODE and use it in out():

out(){ 
#     if [[ "$BASH_VERSINFO" -eq 4 ]]; then
          "$CONSOLE_MODE" && printf -- "%b" "${1//%/%%}"
#     else
#          /usr/bin/printf -- "${1//%/%%}"
#     fi
}

Still i have some console output, so i assume it is not everywhere consistent and maybe it should be adapted to out() or just flag "$CONSOLE_MODE" && .. should be set to lines with output.

For example hier:

"$CONSOLE_MODE" && printf -- " %-30s %s" "${cipher[i]}:" "${proto[i]}"

2. I think a JSON or CSV report should be created. If Nagios triggers alarm, it would be nice to look at report to get information what kind of vulnerability it was.

So the calculation will be occurred in fileout(). The format of counter is simple: "0 0 0 0"

fileout() { # ID, SEVERITY, FINDING, CVE, CWE, HINT, VULN_COUNT
     local severity="$2"
     local cwe="$5"
     local hint="$6"
     local counter=${7:-"0 0 0 0"}  # LOW MEDIUM HIGH CRITICAL

     if show_finding "$severity"; then
         local finding=$(strip_lf "$(newline_to_spaces "$(strip_quote "$3")")")
         $NAGIOS && counter=$(count_vulnerabilities "$severity" "$counter")
         ....
         $NAGIOS && ([[ "$severity" == "LOW" ]] || [[ "$severity" == "MEDIUM" ]] || [[ "$severity" == "HIGH" ]] || [[ "$severity" == "CRITICAL" ]]) && echo "$counter"
         "$FIRST_FINDING" && FIRST_FINDING=false
     fi
}

3. New functionality:

#################### NAGIOS ####################

count_vulnerabilities() {
   local severity=$1
   read low_counter medium_counter high_counter critical_counter <<< $2

   if [[ "$severity" == "LOW" ]]; then
           low_counter=$((low_counter+1))
   elif [[ "$severity" == "MEDIUM" ]]; then
           medium_counter=$((medium_counter+1))
   elif [[ "$severity" == "HIGH" ]]; then
           high_counter=$((high_counter+1))
   elif [[ "$severity" == "CRITICAL" ]]; then
           critical_counter=$((critical_counter+1))
   fi

   echo "$low_counter $medium_counter $high_counter $critical_counter"
}

combine_vulnerability_reports() {
   read low_counter1 medium_counter1 high_counter1 critical_counter1 <<< $1
   read low_counter2 medium_counter2 high_counter2 critical_counter2 <<< $2
   low_counter1=${low_counter1:-0}
   low_counter2=${low_counter2:-0}
   medium_counter1=${medium_counter1:-0}
   medium_counter2=${medium_counter2:-0}
   high_counter1=${high_counter1:-0}
   high_counter2=${high_counter2:-0}
   critical_counter1=${critical_counter1:-0}
   critical_counter2=${critical_counter2:-0}
   echo "$((low_counter1 + low_counter2)) $((medium_counter1 + medium_counter2)) $((high_counter1 + high_counter2)) $((critical_counter1 + $critical_counter2))"
}

print_vulnerability_assessment_report() { # LOW MEDIUM HIGH CRITICAL  ##### not tested yet.
   local warning=$((LOW+MEDIUM))
   local critical=$((HIGH+CRITICAL))
   local state="UNKNOWN"
   local exitcode=0

   if [[ $warning -gt 0 ]]; then
        state="CRITICAL"
        exitcode=2
   elif [[ $critical -gt 0 ]]; then
        state="WARNING"
        exitcode=1
   else
        state="OK"
        exitcode=0
   fi

   printf "STATE:%s [LOW:%s, MEDIUM:%s, HIGH:%s, CRITICAL:%s]" "$state" "$LOW" "$MEDIUM" "$HIGH" "$CRITICAL"
   exit $exitcode
}

4. How it will be used:

We are interested only in severities LOW, MEDIUM, HIGH, CRITICAL. For example run_heartbleed:

run_heartbleed(){
     ....
     local counter="0 0 0 0"
     ...
     pr_svrty_critical "VULNERABLE (NOT ok)"
     counter=$(fileout "heartbleed" "CRITICAL" "Heartbleed: VULNERABLE $append" "$cve" "$cwe" "$hint" "$counter") 
     ...
     $NAGIOS && echo "$counter"
     return $ret
}

Hier as i posted already, i am not sure what to use.

lets_roll() function:

# something like this:
fileout_section_header $section_number true && ((section_number++)) && FIRST_FINDING=true
     if [[ $do_heartbleed ]]; then
        report="$(run_heartbleed)"
        ret=$(($? + ret))
        echo "$ret"
        vuln_counter=$(combine_vulnerability_reports "$vuln_counter" "$report") && FIRST_FINDING=false
     fi

# or this one variant:
$do_ccs_injection && vuln_counter=$(combine_vulnerability_reports "$vuln_counter" "$(run_ccs_injection)")
	 echo "REPORT: $vuln_counter"

Somewhere at the end should be called print_vulnerability_assessment_report() and result should be output: "STATE:CRITICAL [LOW:1, MEDIUM:0, HIGH:5, CRITICAL:1]"

---

So to discuss:

• this approach at all
• what to do with "ret"
• maybe we should collect statistic every time and only showing by --nagios
• json or json-pretty as default for --nagios
• nagios message: ("STATE:CRITICAL [LOW:1, MEDIUM:0, HIGH:5, CRITICAL:1]") this one or something else
• set "$CONSOLE_MODE &&" to lines that don't use out() and direct call printf... or make them calling out() (it was about 8-10 places, so i am for the second option)
• how to calculate warning and critical for nagios. Right now i observe low and medium as warning and high and critical as critical.
• anything else?

I also would do a similar thing for a rating.

i did not understand.

[drwetter]

Sorry, I overlooked this one -- notifcation ended up for some reason in my Junk folder :(

I like the idea with the counter. My suggestion:

• ignore $ret
• for fileout() arg[3] is always the severity. As it is always being called: how about calling a function in fileout() which does a similar thing as count_vulnerabilities().
• then I would strictly separate count CRITICAL, HIGH etc.
• internally the counter could be like counter_high, counter_critical or an array, e.g. severity_counter[0] => info, severity_counter[1] => low, severity_counter[2] => medium etc... . That variable should be global (too lazy now to correct them to UPPERCASE)

Further comments / questions?

• I didn't get the thing with the "$CONSOLE_MODE
• does the console output have to be muted for icinga/ nagios?
• do we need `--nagios`` at all?

[drwetter]

You should be able to check at least whether a run encountered an error or not. Please see commit message (a0dabf9)

[gjedeer]

Just a workaround, but this works for Nagios:

#!/bin/bash

set -e 

json_file=$(mktemp)

./testssl.sh -oJ $json_file $@ | logger -t testssl
CRIT=$(jq '.. | .severity?' $json_file | grep CRITICAL | wc -l)
rm $json_file
if [ $CRIT -gt 0 ]; then 
    echo "$CRIT TLS issues"
    exit 1
else
    echo "OK"
    exit 0
fi

[drwetter]

cool workaround! Thx for sharing

[GottemHams]

I actually came across this issue when I was looking for something else. I wrote an icinga2 check plugin that uses testssl.sh which might be of use to the people who commented here (and perhaps the people from testssl too).

Do note though that it's a little crude as it was originally intended to be used by just myself. Also I've decided to output low severity stuff too, as offering TLSv1.0 is classified as such but I feel you should be warned about it as well. ;]

[drwetter]

Okay. thanks for sharing! Maybe others find it useful too!

[dnmvisser]

Hi i've created one as well https://github.com/dnmvisser/nagios-testssl

[drwetter]

FYI (@GottemHams / @dnmvisser): I added both links to the Readme.md at the entry URL.

[brian-villanueva]

For those interested in using jq to parse out the results (as @gjedeer suggested above as a workaround), you can do that without grep and wc:

$ cat myfile.json | jq 'group_by(.severity) | .[] | { "key" : (.[0].severity), "value": (. | length) }' | jq -s 'from_entries'
{
  "CRITICAL": 3,
  "HIGH": 1,
  "INFO": 106,
  "LOW": 9,
  "MEDIUM": 2,
  "OK": 31
}

[drwetter]

The handling of trivy seems an easy thing to apply to testssl.sh:

trivy <whatever> --exit-code 1 --severity <SEVERITY> <SCANTARGET>

Surprisingly testssl.sh has the very same severity switch already. :-) It just needs the --exit-code switch and a way to track whether there was a severity of level X or greater (trivy lists same=equal, not greater-equal as). That doesn't seem complicated.

If anybody feels like implementing that pls let me know.