pgsync
pgsync copied to clipboard
when using through_tabes, New data added in Postgres is not getting sync to elasticsearch
PGSync version: 2.1.1
Postgres version: 13.2
Elasticsearch version: 7.6
Redis version: 6.2.2
Python version: 3.7.10
Problem Description:
The initial synchronization of pgsync was successful. The added data was not synchronized to the ES, and there was no error. When I used comment as only children, it synchronize success. I can now add a channel child and use through_tables is associated with tables. When adding data, it is not synchronized to the ES, and there is no error message. I restart pgsync, and it can synchronize all, but it is still not synchronized to the es when adding data; I tried to associate only one children in the channel, but failed to add data. It can be seen use through_tables as associated, the synchronization is unsuccessful and no error is reported.
schema.json is as follows:

Database table posts_channel is an association table automatically created using orm
Please look at this problem. It's urgent.
Error Message (if any):
After adding data, no error is reported, but the data is not synchronized to es.

I'm having somewhat of a similar issue.
If I add a relationship with "through_tables" the sync will not run, it just lists out the different table names and creates the hidden checkpoint file. The minute I remove the child with the through_tables value it works fine.
I've deleted the index, re-run the bootstrap and deleted the checkpoint file, but it will not sync the data unless I remove the through_tables realationship.
- What do you mean by the sync will not run.
- Looks to me like something is wrong with the way you have defined the through_table
- If you change the structure of the schema.json you will need to remove the checkpoint file and re-index.
- Do you have a foreign key between the posts_channel correctly.
- If this is urgent please provide your database schema dump (no-data) and I will try to reproduce.
After adding through_table, adding new data is not synchronized to es, restarting pgsync will synchronize all the data in the database to es. I deleted the index in es, and the posts_channel foreign key is correct. I currently use the one-to-many association posts_channel method.
To achieve the purpose by querying the fields in the associated posts_channel
Adding an explicit foreign_key to your schema implies that you don't have the foreign key constraints defined in your database.
I would still like to see your database schema to get a better understanding.
Your first schema did not have foreign_key but this one does.
Adding foreign_key in the relationship is required ONLY when your database does do define this relationship between the tables.
I am facing the same issue for one-to-many relationships where adding records are not being synced but edit and deletes are working fine. Are there any updates on this?
@Dipesh-Aspire Can you open a new request with some details please
@toluaina It seems I got the similar thread here: https://github.com/toluaina/pgsync/issues/182
posting my details there.