logisland icon indicating copy to clipboard operation
logisland copied to clipboard

Should remove avro.output.schema line from the index apache logs tutorial config file

Open mathieu-rossignol opened this issue 7 years ago • 0 comments

In the index-apache-logs.yml config file, the stream for the SplitText processor has the following config key: avro.output.schema: > { "version":1, "type": "record", "name": "com.hurence.logisland.record.apache_log", "fields": [ { "name": "record_errors", "type": [ {"type": "array", "items": "string"},"null"] }, { "name": "record_raw_key", "type": ["string","null"] }, { "name": "record_raw_value", "type": ["string","null"] }, { "name": "record_id", "type": ["string"] }, { "name": "record_time", "type": ["long"] }, { "name": "record_type", "type": ["string"] }, { "name": "mail_text", "type": ["string","null"] }, { "name": "http_method", "type": ["string","null"] }, { "name": "param_user", "type": ["string","null"] }, { "name": "http_query", "type": ["string","null"] }, { "name": "http_version","type": ["string","null"] }, { "name": "mail_html", "type": ["string","null"] }, { "name": "identd", "type": ["string","null"] }, { "name": "user", "type": ["string","null"] } ]}

In reality, the serialization occurs by using only string, that is any record type you define as long boolean or whatever type ends up at the end as a string in the record outgoing from the stream. Thus, this config line is not used and it's confusing. I think one should remove it.

mathieu-rossignol avatar Mar 13 '17 15:03 mathieu-rossignol