Slow insert after foreign key error

[drjackild]

I start an import of 900MB CSV file into AWS RDS Postgres instance. The first batch is going smoothly without any problem (around 30k rows) but after an error about the Foreign Key constraint, the ingestion speed dropped dramatically to 20-30 rows instead of 25k. Any idea why this happened? Thank you for your help!

PGLoader version 3.6.7

• [ ] did you test a fresh compile from the source tree?

  No, only brew version

• [x] did you search for other similar issues?

  Yes, didn't find a similar problem

LOAD CSV
   FROM 'source.csv'
        HAVING FIELDS
        (
           a, b, c, d
        )
   INTO postgresql://user:padd@localhost:35432/db
        TARGET TABLE public.source
   WITH truncate,
        fields terminated by ','

• [x] pgloader output you obtain

2022-08-22T09:04:14.043781+01:00 LOG pgloader version "3.6.7"
2022-08-22T09:04:14.047954+01:00 LOG Data errors in '/private/tmp/pgloader/'
2022-08-22T09:04:14.048718+01:00 LOG Parsing commands from file #P"/Users/drjackild/projects/pg_loader_load.load"
2022-08-22T09:04:33.813161+01:00 ERROR PostgreSQL ["\"public\".\"source\""] Database error 23503: insert or update on table "source" violates foreign key constraint "source_user_id_fkey"
DETAIL: Key (user_id)=(e6b3f497-295c-4d8f-9aa5-580892de4f5f) is not present in table "user".