meilisync icon indicating copy to clipboard operation
meilisync copied to clipboard

Postgres timestamp column causes sync to fail

Open brandantck opened this issue 2 years ago • 7 comments

Postgres: v15

config.yml

progress: 
  type: file 

I am trying to sync a table from a Postgres database with the column type:timestamp, and its throwing this error. image

This is an example of a row in the database image

Seems like the error occurs when its trying to do a json.dumps into the progress.json file.

Edit: Tried with redis but still getting the same error, issue might not lie with writing to progress

brandantck avatar Oct 26 '23 09:10 brandantck

datetime is not json serializable so that is most likely causing the issue. I'm not sure the full context here so this my not be possible but if you can convert your dates to an iso string before dumping it should fix the issue.

An example of how to convert:

condition_start_datetime = datetime.datetime(2010, 4, 1, 0, 0)
converted = condition_start_datetime.isoformat()

sanders41 avatar Oct 26 '23 18:10 sanders41

Maybe you can use plugins

long2ice avatar Oct 27 '23 02:10 long2ice

When i run the meilisync refresh command, it doesnt seem to run the plugin pre_event or post_event functions

brandantck avatar Oct 27 '23 07:10 brandantck

Yes, that's a problem...

long2ice avatar Oct 27 '23 07:10 long2ice

Ah okay, so currently, the plugins are only loaded and used on the meilisync start command?

brandantck avatar Oct 27 '23 08:10 brandantck

You can try latest code now

long2ice avatar Nov 29 '23 02:11 long2ice

Hello the postgres timestamp with time zone got converted to unix format for initial data. but the format is corrected after I update the date in the database. So the timestamp is only work correctly for newly inserted/updated data.

I use long2ice/meilisync:dev docker image.

For initial data image

After update image Note: I only update the date of the second item.

Is there any new settings need to be set?

sokhuong-uon avatar Aug 19 '24 03:08 sokhuong-uon