pg_duckdb icon indicating copy to clipboard operation
pg_duckdb copied to clipboard

Resource leak on error, eventual crash on out of memory

Open hlinnaka opened this issue 1 year ago • 2 comments

If an error happens during execution, some resources are leaked. If you keep doing that for long enough, the server crashes:

$ psql postgres -c "create table foo as select 'not an integer' as t"
SELECT 1
$ for run in {1..100000}; do echo "begin; select t::integer from foo; rollback;"; done | psql postgres   2>&1 | tail
                ^
ROLLBACK
BEGIN
ERROR:  Duckdb execute returned an error: Out of Memory Error: Failed to allocate block of 32768 bytes (bad allocation)
ROLLBACK
BEGIN
server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.
connection to server was lost
2024-08-14 11:12:37.122 EEST [1393307] STATEMENT:  select t::integer from foo;
<jemalloc>: Error in munmap(): Cannot allocate memory
<jemalloc>: Error in munmap(): Cannot allocate memory
<jemalloc>: Error in munmap(): Cannot allocate memory
<jemalloc>: Error in munmap(): Cannot allocate memory
<jemalloc>: Error in munmap(): Cannot allocate memory
<jemalloc>: Error in munmap(): Cannot allocate memory
<jemalloc>: Error in munmap(): Cannot allocate memory
<jemalloc>: Error in munmap(): Cannot allocate memory
<jemalloc>: Error in munmap(): Cannot allocate memory
<jemalloc>: Error in munmap(): Cannot allocate memory
<jemalloc>: Error in munmap(): Cannot allocate memory
<jemalloc>: Error in munmap(): Cannot allocate memory
<jemalloc>: Error in munmap(): Cannot allocate memory
<jemalloc>: Error in munmap(): Cannot allocate memory
<jemalloc>: Error in munmap(): Cannot allocate memory
<jemalloc>: Error in munmap(): Cannot allocate memory
<jemalloc>: Error in munmap(): Cannot allocate memory
2024-08-14 11:12:37.125 EEST [1393307] ERROR:  Duckdb execute returned an error: Out of Memory Error: Failed to allocate block of 32768 bytes (bad allocation)
2024-08-14 11:12:37.125 EEST [1393307] STATEMENT:  select t::integer from foo;
2024-08-14 11:12:37.417 EEST [1188075] LOG:  server process (PID 1393307) was terminated by signal 11: Segmentation fault
2024-08-14 11:12:37.417 EEST [1188075] DETAIL:  Failed process was running: select t::integer from foo;
2024-08-14 11:12:37.417 EEST [1188075] LOG:  terminating any other active server processes
2024-08-14 11:12:37.418 EEST [1188075] LOG:  all server processes terminated; reinitializing

hlinnaka avatar Aug 14 '24 08:08 hlinnaka

I'm pretty sure this is caused by to #93, but good to have an easily reproducable case.

JelteF avatar Aug 19 '24 09:08 JelteF

I'm pretty sure this is caused by to https://github.com/duckdb/pg_duckdb/issues/93, but good to have an easily reproducable case.

Yeah, perhaps.

There are two things to investigate here really:

  1. The leak itself. Leaking is bad.
  2. Why does the system segfault after running out of memory?

hlinnaka avatar Aug 20 '24 08:08 hlinnaka