matano
matano copied to clipboard
Bug when defining a custom log source with all custom table configurations
Bug brought up in discord channel:
Was popping in because I had a quick question. Is it possible to configure tables with a custom log source? Couldn’t seem to get that working and when walking through the code
My setup looks something like this:
# /log_sources/customsource/log_source.yml
name: customsource
ingest:
select_table_from_payload_metadata: |
if match(.__metadata.s3.key, r'.*') {
"mytable"
}
expand_records_from_payload: "parse_json!(.__raw)"
schema:
ecs_field_names:
- event.original
transform: |
.event.original = encode_json(.json)
# /log_sources/customsource/tables/mytable.yml
name: "mytable"
schema:
fields:
- name: custom_field
type: string
I expected it to create a mytable.yml schema, but instead it creates default.yml and ignores any configurations in my table.
The log-source.ts construct needs to be updated to properly pick up user configuration for custom log sources / tables.