pg_bulkload icon indicating copy to clipboard operation
pg_bulkload copied to clipboard

WARNING: temporary file leak (PG13)

Open roymcmorran opened this issue 4 years ago • 1 comments

At the end of a pg_bulkload operation many warnings are generated about temporary file leaks:

$ pg_bulkload ref.txt.ctl --dbname xxxx --host xxxx --port 5432 --username load < ref.txt
Password: 
NOTICE: BULK LOAD START
WARNING:  temporary file leak: File 28 still referenced
WARNING:  temporary file leak: File 66 still referenced
...
...65 more...
...
WARNING:  temporary file leak: File 38 still referenced
WARNING:  temporary file leak: File 56 still referenced
NOTICE: BULK LOAD END
	1 Rows skipped.
	281901397 Rows successfully loaded.
	0 Rows not loaded due to parse errors.
	0 Rows not loaded due to duplicate errors.
	0 Rows replaced with new rows.

These seem to refer to temp files that are created by the postmaster process under $PGDATA/base/pgsql_tmp At the end of the load all the temp files are removed (despite the warning) and it appears that the load completed successfully (based on a simple row count).

Note that this only happens if the table has existing rows. If you truncate the table first (for testing) the warnings do not appear.

I see from the PostgreSQL source code that these warnings come from the 'Resource Owner' code (see https://github.com/postgres/postgres/tree/master/src/backend/utils/resowner ). However the warnings only appear during the pg_bulkload operation.

We are migrating from PostgreSQL 9.5 and pg_bulkload 3.16 to the latest versions. These processes work as expected with no warnings in the old environment.

$ pg_bulkload --version
pg_bulkload 3.1.17
$ psql --version
psql (PostgreSQL) 13.2

Thank you. ref.txt.ctl.txt

roymcmorran avatar Feb 23 '21 23:02 roymcmorran

This issue is still present with 3.1.19 and PG14. Any ideas? Thanks.

roymcmorran avatar Dec 10 '21 15:12 roymcmorran