Kansa Core: Add ability to directly store output in Azure

[jt-msft]

Logging data to the hard drive of the analysis host is suitable for most cases, but when dealing with very large numbers of targets or when a web request is the only quick way to push results out of the network, a direct-to-web option would be useful.

Benefits:

• Results are immediately off-host and more difficult to tamper.
• Azure storage is cheap and massive.
• Azure offers additional analysis engines, like HDInsight.

Requirements:

• Override #OUTPUT directive to always output XML (or PSObjects?).
• Certificate pinning.
• Need server-side parsing engine (probably as a separate project).
• Will still need a local cache for fall back in case the web requests fail.

[davehull]

I don't like XML as an output, so I'd recommend making that optional. XML is slow and painful for analysis. Maybe a JSON option should be added to the framework and for this?

On Tue, Aug 19, 2014 at 9:38 AM, Jon [email protected] wrote:

Logging data to the hard drive of the analysis host is suitable for most cases, but when dealing with very large numbers of targets or when a web request is the only quick way to push results out of the network, a direct-to-web option would be useful.

Benefits:

• Results are immediately off-host and more difficult to tamper.
• Azure storage is cheap and massive.
• Azure offers additional analysis engines, like HDInsight.

Requirements:

• Override #OUTPUT directive to always output XML (or PSObjects?).
• Certificate pinning.
• Need server-side parsing engine (probably as a separate project).
• Will still need a local cache for fall back in case the web requests fail.

— Reply to this email directly or view it on GitHub https://github.com/davehull/Kansa/issues/61.