quickwit
quickwit copied to clipboard
Support specifying a timezone manually with strptime parser
For instance:
name: timestamp
type: datetime
description: Time at which the event was emitted
input_formats:
- rfc3339
- unix_timestamp
- "%Y %m %d %H:%M:%S.%f %z"
- strptime:
format: "%Y %m %d %H:%M:%S.%f"
default_timezone: America/New_York
Maybe default_timezone?
default_timezone sounds better, indeed.
I think we should raise an error during the parsing of the config if the user provides a format with a timezone and a default timezone (%z or %Z). It's better to fail early. Any opinion @fulmicoton ?
Parsing a timezone like America/New_York and use it to compute a timestamp from a date string requires a TZ database. The most popular / better maintained one seems to be chrono-tz, but we use time almost everywhere in Quickwit (and Tantivy). There is a time-tz crate that would fit more nicely into the existing code, but it appears to be substantially less used.
In #5248 I'm rewriting a tiny bit the code that parse dates. It will be possible to pass an utc offset which might simplify this issue. Getting the offset from the timezone will require a bit of work/wits however.
You may also have a look at https://github.com/quickwit-oss/quickwit/tree/guilload/strptime-timezone