Confusion about json_t

[rmouritzen-splunk]

The json_t type has ended up with two interpretations:

Options

Option 1: json_t is the any type

• The json_t is an any type, meaning any valid JSON type can be placed here.
• When OCSF events are expressed as JSON, attributes of this value are the any valid JSON type.
• When OCSF events are expressed in serialization formats like Protobuf, a variant type would need to be defined and used, or the valid would need to be expressed in JSON. Either way, this is somewhat complicated.

Option 2: json_t is a string containing JSON

• The json_t is a string that is a JSON encoding of any valid JSON.
• When OCSF events are expressed as JSON, attributes of this type are double encoded.
• When OCSF events are expressed in serialization formats like Protobuf, attributes of this type are a string containing JSON.

Note

The original intent was json_t is an any type. However actually usage has ended up split between the two interpretations above. Changing to either interpretation will cause pain for some members of the community, so both interpretations should be up for consideration going forward.

[lukas-krecan-s1]

Just for context, after long discussion we are encoding json_t to google.protobuf.Struct in SentinelOne Technically it does not cover "primitive" JSON values (like a single number in the root) but it seemed to be a lesser evil. In other words, we defined json_t as any JSON object.

[rmouritzen-splunk]

Defining json_t as a JSON object actually wouldn't work for at least one of Splunk's internal use-cases. We have a field that ends up as a string in one case and an object in another. (These are both from Windows Event Logs. The first case comes from XML events, while the other comes from the older multiline events.)

[rmouritzen-splunk]

Marked with the "v2.0 and later" tag since there's not much that can be done to the 1.x schema to fix this.

[rmouritzen-splunk]

For the 1.x schemas, we'll clarify the description with the original "any type" intent.