observability icon indicating copy to clipboard operation
observability copied to clipboard

[FEATURE] Create an ECS integration to SS4O

Open derek-ho opened this issue 1 year ago • 3 comments

Security Dashboards is tightly following Elastic's ECS schema - if we have an integration for ECS it will help with security analytics integration use case, as well as folks who are transferring data from elastic to opensearch

derek-ho avatar Mar 20 '23 17:03 derek-ho

are you referring for AWS - ECS integration ?

YANG-DB avatar Mar 20 '23 22:03 YANG-DB

Updated description - I am talking about elastic's ECS https://www.elastic.co/guide/en/ecs/current/index.html

derek-ho avatar Mar 20 '23 22:03 derek-ho

I think we should approach this in a gradual manner - Integration by Integration ... As soon as we are presented with a new log-based Integration request - it should also be accompanied with the appropriate catalog-component representing its fields content...

Currently our schema catalog approach supports incrementally adding new log type support which formulate a components sub categories under the category section of the schema.

Each of these component (sub-categories) represent some scope of fields that address the relevant domain they are associated to (cloud, registry, file-system, Network ...)

NginX example: For the first Integration of the NginX web server, we added the http.mapping support to reflect the needs for the Nginx integration for http based log fields

In a similar manner, once we will be introduced with a requirement for K8 Integrations - for example, we will add the appropriate compound fields mapping to the Observability logs folder and thus allow the Integration template to explicitly declare its dependency on that component.

This mechanism helps the Integrations loading process to know which compound template it need to add to the root log template sso_logs_mapping template .

Once the template generation is complete - it will be able to support the integration's data-stream ingested documents

YANG-DB avatar Mar 20 '23 22:03 YANG-DB