Nested optional fields in JSON lead to unparsed events

[landauermax]

The fields event_data and TargetImage are both optional in the config parser:

LearnMode: True

LogResourceList:
    - 'file://home/ubuntu/test.yml'

Parser:
  - id: event_id
    type: VariableByteDataModelElement
    name: 'event_id'
    args: '0123456789'

  - id: computer_name
    type: AnyByteDataModelElement
    name: 'computer_name'

  - id: ts
    type: DateTimeModelElement
    name: 'timestamp'
    date_format: "%Y-%m-%dT%H:%M:%S.%fZ" # 2024-03-21T20:46:06.955Z

  - id: target_image
    type: AnyByteDataModelElement
    name: 'target_image'

  - id: source_image
    type: AnyByteDataModelElement
    name: 'source_image'

  - id: json
    start: True
    type: JsonModelElement
    name: 'model'
    allow_all_fields: True
    optional_key_prefix: "*"
    key_parser_dict:
      "@timestamp": ts
      winlog:
        "*event_data":
          "*TargetImage": target_image
        event_id: event_id
        computer_name: computer_name

Input:
        timestamp_paths: "/model/@timestamp/timestamp"
        json_format: True

Analysis:
   - type: 'VerboseUnparsedAtomHandler'
     id: vuah

   - type: NewMatchPathDetector
     id: NewMatchPathDetector
     suppress: True
     learn_mode: True

EventHandlers:
    - id: "stpe"
      type: "StreamPrinterEventHandler"
      json: True

When the log line has only event_data but not TargetImage, the event will be unparsed:

{
  "@timestamp": "2024-04-29T09:54:21.175Z",
  "@metadata": {
    "beat": "winlogbeat",
    "type": "_doc",
    "version": "8.12.2"
  },
  "winlog": {
    "api": "wineventlog",
    "event_data": {
      "TaskName": "\\Microsoft\\Windows\\GroupPolicy\\{A7719E0F-10DB-4640-AD8C-490CC6AD5202}",
      "Path": "gpupdate.exe",
      "ProcessID": "396",
      "Priority": "32"
    },
    "event_id": "129",
    "task": "Created Task Process",
    "provider_guid": "{de7b24ea-73c8-4a09-985d-5bdadcfa9017}",
    "channel": "Microsoft-Windows-TaskScheduler/Operational",
    "provider_name": "Microsoft-Windows-TaskScheduler",
    "record_id": 40556,
    "user": {
      "name": "SYSTEM",
      "type": "Well Known Group",
      "identifier": "S-1-5-18",
      "domain": "NT AUTHORITY"
    },
    "process": {
      "thread": {
        "id": 4020
      },
      "pid": 2144
    },
    "computer_name": "atb-client01.aecid-testbed.com",
    "opcode": "Info"
  },
  "event": {
    "created": "2024-04-29T09:58:37.236Z",
    "code": "129",
    "kind": "event",
    "provider": "Microsoft-Windows-TaskScheduler",
    "action": "Created Task Process"
  },
  "log": {
    "level": "information"
  },
  "host": {
    "name": "atb-client01.aecid-testbed.com"
  },
  "tags": [
    "WEC2-Task-Scheduler"
  ],
  "ecs": {
    "version": "8.0.0"
  },
  "agent": {
    "version": "8.12.2",
    "ephemeral_id": "f50bc076-6d30-4188-b8a6-2707dd4256ab",
    "id": "284c17a5-4437-49a8-bf15-96e03c4c1bd2",
    "name": "wec",
    "type": "winlogbeat"
  }
}

The event should be able to be parsed, Removing TargetImage field from the parser defined in the config, the log line can be parsed. I assume this is a bug in the JSON parser.