pgtoolkit icon indicating copy to clipboard operation
pgtoolkit copied to clipboard

Relax date format requierment (UTC)

Open blogh opened this issue 4 years ago • 3 comments

Hi,

https://github.com/dalibo/pgtoolkit/blob/249034a214cddc223b7b7d4162f056ceb93899ca/pgtoolkit/log/parser.py#L128

Would it be possible to have an option to bypass this requirement ? I wanted to use this class to handle log parsing in a project where I just needed the date "as is" and couldn't because of this.

blogh avatar Oct 18 '21 13:10 blogh

Looks reasonable. By the way, it's not clear to me why datetime.strptime() is not used directly here.

dlax avatar Oct 18 '21 14:10 dlax

By the way, it's not clear to me why datetime.strptime() is not used directly here.

maybe it's because there are several formats in log_line_prefix ?

https://www.postgresql.org/docs/14/runtime-config-logging.html#GUC-LOG-LINE-PREFIX

  # %m (Time stamp with milliseconds)+ log_timezone = Europe/Paris
>>d = datetime.datetime.strptime("2021-09-22 19:00:45.894 CET", "%Y-%m-%d %H:%M:%S.%f %Z")
  
  # %t (Time stamp without milliseconds)
>>d = datetime.datetime.strptime("2021-12-07 10:14:04 CET", "%Y-%m-%d %H:%M:%S %Z")

--# %n (Time stamp with milliseconds (as a Unix epoch)) (not supported in pgtoolkit, never seen it, do we want it ?)
>>d = datetime.datetime.fromtimestamp(1638868691.099)

blogh avatar Dec 07 '21 09:12 blogh

We could try different parsing methods (strptime, or even datetime.fromisoformat() when there is no tz info). Another option would be to use https://dateutil.readthedocs.io/en/stable/ but that would pull a dependency (the first, IIRC).

@blogh, if you need something to be changed in some way, feel free to submit a patch.

dlax avatar Dec 14 '21 10:12 dlax