quickwit icon indicating copy to clipboard operation
quickwit copied to clipboard

Support specifying a timezone manually with strptime parser

Open guilload opened this issue 2 years ago • 6 comments

For instance:

name: timestamp
type: datetime
description: Time at which the event was emitted
input_formats:
  - rfc3339
  - unix_timestamp
  - "%Y %m %d %H:%M:%S.%f %z"
  - strptime:
      format: "%Y %m %d %H:%M:%S.%f"
      default_timezone: America/New_York

guilload avatar Apr 12 '23 15:04 guilload

Maybe default_timezone?

fulmicoton avatar Apr 20 '23 02:04 fulmicoton

default_timezone sounds better, indeed.

guilload avatar May 15 '23 15:05 guilload

I think we should raise an error during the parsing of the config if the user provides a format with a timezone and a default timezone (%z or %Z). It's better to fail early. Any opinion @fulmicoton ?

rdettai avatar Jul 24 '24 12:07 rdettai

Parsing a timezone like America/New_York and use it to compute a timestamp from a date string requires a TZ database. The most popular / better maintained one seems to be chrono-tz, but we use time almost everywhere in Quickwit (and Tantivy). There is a time-tz crate that would fit more nicely into the existing code, but it appears to be substantially less used.

rdettai avatar Jul 24 '24 15:07 rdettai

In #5248 I'm rewriting a tiny bit the code that parse dates. It will be possible to pass an utc offset which might simplify this issue. Getting the offset from the timezone will require a bit of work/wits however.

fulmicoton avatar Aug 02 '24 02:08 fulmicoton

You may also have a look at https://github.com/quickwit-oss/quickwit/tree/guilload/strptime-timezone

guilload avatar Aug 02 '24 10:08 guilload