[SIEM integration] Collect information about runtime dependencies

[simitt]

As layed out in the design doc information about runtime dependencies could be relevant information to show within the SIEM app. Required design steps are:

• [x] Define field set for software package runtime dependencies in ECS https://github.com/elastic/ecs/issues/515
• [ ] create mock ups for SIEM integration
• [ ] investigate which information can be collected from the agents and with how much effort

[simitt]

WIP branch for defining ECS fields: https://github.com/elastic/ecs/pull/532

Initial draft for visualization:

Initial draft for event of type package (updated with information provided by @SergeyKleyman ):

{
  "_source": {
    "agent" : {
      "name" : "elastic-dotnet",
      "version" : "1.0.0"
    },
    "process" : {
      "pid" : 1234,
      "title" : "dotnet",
      "ppid" : 6789
    },
    "processor" : {
      "name" : "package",
      "event" : "package"
    },
    "labels" : {
      "tag1" : "one",
      "tag2" : 2
    },
    "observer" : {
      "hostname" : "apm-server-host",
      "id" : "25e0260e-973e-4f8c-af0d-c0b192618f34",
      "type" : "apm-server",
      "ephemeral_id" : "0fec2346-fd58-47e4-83bc-eb5c1622da68",
      "version" : "8.0.0",
      "version_major" : 8
    },
    "@timestamp" : "2019-08-27T13:28:15.535Z",
    "ecs" : {
      "version" : "1.1.0"
    },
    "service" : {
      "environment" : "staging",
      "framework" : {
        "name" : "xyz",
        "version" : "1.2.3"
      },
      "name" : "1234_service-12a3",
      "runtime" : {
        "name" : "dotnet",
        "version" : "8.0.0"
      },
      "language" : {
        "name" : "c#",
        "version" : "8"
      },
      "version" : "5.1.3"
    },
    "host" : {
      "hostname" : "node-name",
      "os" : {
        "platform" : "darwin"
      },
      "ip" : "127.0.0.1",
      "name" : "node-name",
      "architecture" : "x64"
    },
    "package": {
      "artifact_id": "1232cc0d83f794f6fc6190f603fa2f62dd9aa2017165d19ff1d62742955a7a05",
      "name": "Newtonsoft.Json"",
      "version": "11.0.2.0",
      "description": " Json.NET is a popular high-performance JSON framework for .NET",
      "size": "656384",
      "installed": "2018-03-24T17:44:15.535Z",
      "url.original": "https://www.nuget.org/api/v2/package/Newtonsoft.Json/11.0.2",
      "checksum": "30ad4fe6b2a6aeed",
      "detailed_version": "11.0.2.0 @BuiltBy: dlab14-DDVSOWINAGE051 @Branch: release/11.0.2-MSRC @SrcCode: https://github.com/JamesNK/Newtonsoft.Json/tree/11.0.2",
      "build_time": "2019-01-08T10:18:15.535Z",
      "license": "MIT",
    }
  }
}

@SergeyKleyman this is far from being ready, but I'd like to use this initial drafts to get your opinion early on, mainly regarding which information should we collect in the agents / is feasible to collect.

Open questions:

• When to collect package information?
• Can we / Does it make sense to collect failed install attempts or is this too far away from monitoring apps and too close to information collected by e.g. auditbeat?
• How to avoid storing the same package information multiple times?
• Should we also consider other dependencies, e.g. file dependencies
• Would it make sense to encode a dependant/dependencies and direct/transient information into the events?