improve qualys import times

[jmartin-tech]

trafficstars

~When importing data from recent Qualys reports many RESULT tags contain extra long lines and significant content that result in extremely long processing times when parsed by REXML. By removing all RESULT tags not currently processed during import the time to process data can be significantly improved.~

~This drops all RESULTS not associated with TCP/UDP service listing prior to parsing the XML from the file.~

EDIT: Revised approach uses Nokigiri::XML document and Xpath for improved performance over REXML without mangling the input data.

I can see lots of possible iteration for improvement of import by processing more RESULT data from the Qualys report however at this time only hosts, services, and vulns are imported and no other parsing is done for RESULT tags.

If anyone has recommendations on a better way to organize the constants defined here please comment, the current change just attempts to adjust the values to a single definition.

Verification

List the steps needed to make sure this thing works

• [ ] Start msfconsole
• [ ] db_import <QUALYS_DATA_FILE>
• [ ] Verify Host / Service / Vuln data imports in to the database with same values as prior to change

[gwillcox-r7]

Cut for brevity n censored for customer safety:

[*] Importing host *redacted*
[*] Importing host *redacted*
[*] Importing host *redacted*
[*] Importing host *redacted*
[*] Successfully imported /home/gwillcox/Desktop/*redacted*.xml
msf6 > 

Took only a few minutes vs the 14 hours or so this data previously claimed to have been taking to import.

[gwillcox-r7]

Additional checks look good. Will get this landed now.

[gwillcox-r7]

Release Notes

The process for importing Qualys scan data has been switched over from REXML to using Nokigiri::XML and XPath for improved performance.