fhir-data-pipes icon indicating copy to clipboard operation
fhir-data-pipes copied to clipboard

Published 20 hours ago • verified publisher icon google

Remove deleted resources when exporting to sink FHIR store and database during incremental run

Open chandrashekar-s opened this issue 2 years ago • 4 comments
trafficstars

During incremental run, the deleted resources have to be deleted from the sink FHIR store path and the database. Currently, as part of the fix #547 these deleted resources will be removed from the merged parquet files.

chandrashekar-s avatar Mar 15 '23 17:03 chandrashekar-s

Can you explain the difference between this and #547 ? Is this bug more about handling the case if the export target is a FHIR store and #547 is about deleting the resource in the case of exporting to parquet file?

jjtswan avatar Mar 17 '23 06:03 jjtswan

Yes, this is just an extension for #547 to handle the case where the export target is a FHIR store or another JDBC store like Postgres. Since these cases were not in immediate priority, created this ticket to be handled later on.

chandrashekar-s avatar Mar 20 '23 05:03 chandrashekar-s

Got it. Thanks for the explanation.

jjtswan avatar Mar 20 '23 18:03 jjtswan

Given that our "database as DWH" story is maturing (especially with support for SQL-on-FHIR-v2 views), this bug is more important to be fixed. There is also some renewed interest in syncing to another FHIR store (e.g., #825) which again is relevant to this bug. So changing the priority accordingly.

bashir2 avatar Feb 09 '24 20:02 bashir2