logstash icon indicating copy to clipboard operation
logstash copied to clipboard

Date filter doesn't work with 8.4.1 but does with 8.3.3

Open chongma opened this issue 2 years ago • 3 comments

When processing a simple httpd combined log 8.4.1 fails with _dateparsefailure tag. 8.3.3 works with exact same config

filter {
    date {
        match => ["timestamp",  "dd/MMM/yyyy:HH:mm:ss Z"]
    }
}

chongma avatar Sep 08 '22 13:09 chongma

8.4.x includes updates to the bundled JDK (from Java 11 to Java 17), which in turn includes updates to localization that could likely affect the parsing of dates and times, especially with localized fragments in the pattern like MMM.

Do you have examples of inputs that are now failing and your system's default locale? You may need to provide a more specific locale to the date filter to override your system's locale and let it find the correct one. Alternatively, we may be able to opt out of the newer Java localization paths by adding -Djava.locale.providers=COMPAT,SPI to your config/jvm.options.

yaauie avatar Sep 08 '22 13:09 yaauie

Examples are "07/Sep/2022:18:31:43 +0100", "07/Sep/2022:18:31:43 +0100" etc. It is the whole log file. I think my default locale should be en-GB. -Djava.locale.providers=COMPAT,SPI in config/jvm.options works. Thanks.

chongma avatar Sep 08 '22 16:09 chongma

Providing locale => "en-GB" to the date filter does not seem to work in my case. UPDATE: locale => "en" DOES seem to work

chongma avatar Sep 08 '22 17:09 chongma