pgstac icon indicating copy to clipboard operation
pgstac copied to clipboard

Added a command to bulk delete items included in an ndjson file.

Open edkeeble opened this issue 2 years ago • 0 comments

Background

Given that uniqueness constraints on the items table combine id and datetime, loading data with the upsert method will fail if the datetime for an existing row has changed (but the id is the same, of course).

I don't have a solution to this issue, but I do have a workaround, which is to bulk delete items prior to loading them. This isn't the most efficient solution, but it will work for my use case. I'm 100% open to alternative approaches and wanted to open this PR to seek feedback.

Approach

I've added a new deleteitems command, which takes an NDJSON file and deletes items from the DB matching the ids of each row in the file. The idea is to use the same input file to bulk delete and then insert data.

Concerns

Aside from the inefficiency of this approach, I wonder if providing an NDJSON file of complete STAC items while deleting based solely on the id could confuse users.

edkeeble avatar Mar 07 '22 19:03 edkeeble