pg_duckdb
pg_duckdb copied to clipboard
duckdb
SIGSEGV when pg_duckdb is used in conjunction with timescale
What happens?
Timescale has a hash table that gets initialized in its planner hook.
https://github.com/timescale/timescaledb/blob/9b2789d704c5db37d008c8aa650f77a750beae8b/src/planner/planner.c#L125-L133
https://github.com/timescale/timescaledb/blob/9b2789d704c5db37d008c8aa650f77a750beae8b/src/planner/planner.c#L617-L637
In this stack trace, you can see that timescale_planner() is non-existent. It seems to be because pg_duckdb is calling standard_planner() directly instead of using the hook.
https://github.com/duckdb/pg_duckdb/blob/6dc835c9ba8e3db3441183dc42d2c13224fe37b6/src/pgduckdb_planner.cpp#L172
I'm sure there is a good reason for this, but it seems to break the assumption that timescale has, which is that its planner hook is called. Here is where the create_upper_paths_hook is defined:
https://github.com/postgres/postgres/blob/0cf205e122ae0fe9333ccf843c2269f13ddc32fc/src/backend/optimizer/plan/planner.c#L75-L76
This issue feels like pg_duckdb breaking an invariant, but I'm not sure, and would like to solicit opinions before I go bug the timescale folks about maybe using more defensive programming. Though I'm not sure if that would help anyway.
1749556806956 2025-06-10T12:00:06.956Z Core was generated by `postgres: squaregps_app_access client_372282 ::1(59722) SEL'.
1749556806956 2025-06-10T12:00:06.956Z Program terminated with signal SIGSEGV, Segmentation fault.
1749556806956 2025-06-10T12:00:06.956Z #0 BaserelInfo_insert_hash_internal (found=<optimized out>, hash=<optimized out>, key=<optimized out>, tb=<optimized out>) at /usr/local/pgsql/include/server/common/hashfn.h:99
1749556806956 2025-06-10T12:00:06.956Z [Current thread is 1 (Thread 0x7f9be9d020 (LWP 19437))]
1749556806956 2025-06-10T12:00:06.956Z #0 BaserelInfo_insert_hash_internal (found=<optimized out>, hash=<optimized out>, key=<optimized out>, tb=<optimized out>) at /usr/local/pgsql/include/server/common/hashfn.h:99
1749556806956 2025-06-10T12:00:06.956Z #1 BaserelInfo_insert (tb=0x0, key=key@entry=114695, found=found@entry=0x7fe57cfdbf) at /usr/local/pgsql/include/server/lib/simplehash.h:781
1749556806956 2025-06-10T12:00:06.956Z #2 0x0000007f6ff6f9bc in get_or_add_baserel_from_cache (parent_reloid=0, chunk_reloid=114695) at /ext-src/timescaledb-src/src/planner/planner.c:775
1749556806956 2025-06-10T12:00:06.956Z #3 ts_classify_relation (root=root@entry=0x7f6895e928, rel=rel@entry=0x5581730590, ht=ht@entry=0x7fe57cfe30) at /ext-src/timescaledb-src/src/planner/planner.c:876
1749556806956 2025-06-10T12:00:06.956Z #4 0x0000007f6ff6fae8 in timescaledb_create_upper_paths_hook (root=0x7f6895e928, stage=UPPERREL_FINAL, input_rel=0x5581730590, output_rel=0x55816e78b8, extra=0x7fe57d0048) at /ext-src/timescaledb-src/src/planner/planner.c:1602
1749556806956 2025-06-10T12:00:06.956Z #5 0x000000556dc02030 in grouping_planner (root=root@entry=0x7f6895e928, tuple_fraction=<optimized out>, tuple_fraction@entry=0, setops=setops@entry=0x0) at planner.c:2031
1749556806956 2025-06-10T12:00:06.956Z #6 0x000000556dc04e10 in subquery_planner (glob=glob@entry=0x55816ef5f8, parse=parse@entry=0x7f6895ed30, parent_root=parent_root@entry=0x0, hasRecursion=hasRecursion@entry=false, tuple_fraction=tuple_fraction@entry=0, setops=setops@entry=0x0) at planner.c:1089
1749556806956 2025-06-10T12:00:06.956Z #7 0x000000556dc05204 in standard_planner (parse=0x7f6895ed30, query_string=<optimized out>, cursorOptions=2048, boundParams=<optimized out>) at planner.c:415
1749556806956 2025-06-10T12:00:06.956Z #8 0x0000007f6d0b4624 in DuckdbPlanNode (parse=parse@entry=0x558147e668, query_string=query_string@entry=0x5581479c48 "\n DROP TABLE IF EXISTS temp_tracking_data_core_table;\n\t\tSELECT * FROM raw_telematics_data.tracking_data_core LIMIT 1;\n\t\t\n\t\tCREATE TEMP TABLE temp_tracking_data_core_table as\n SELECT *\n "..., cursor_options=cursor_options@entry=2048, bound_params=bound_params@entry=0x0, throw_error=throw_error@entry=true) at ../../src/pgduckdb/pgduckdb_planner.cpp:176
1749556806956 2025-06-10T12:00:06.956Z #9 0x0000007f6d0c8d58 in DuckdbPlannerHook_Cpp (bound_params=0x0, cursor_options=2048, query_string=0x5581479c48 "\n DROP TABLE IF EXISTS temp_tracking_data_core_table;\n\t\tSELECT * FROM raw_telematics_data.tracking_data_core LIMIT 1;\n\t\t\n\t\tCREATE TEMP TABLE temp_tracking_data_core_table as\n SELECT *\n "..., parse=0x558147e668) at ../../src/pgduckdb/pgduckdb_hooks.cpp:232
1749556806956 2025-06-10T12:00:06.956Z #10 pgduckdb::__CPPFunctionGuard__<PlannedStmt* (*)(Query*, char const*, int, ParamListInfoData*), DuckdbPlannerHook_Cpp, Query*, char const*, int, ParamListInfoData*> (func_name=0x7f6f0d1c88 "DuckdbPlannerHook") at ../../src/pgduckdb/pgduckdb_hooks.cpp:265
1749556806956 2025-06-10T12:00:06.956Z #11 0x000000556dcd6760 in pg_plan_query (querytree=0x558147e668, query_string=0x5581479c48 "\n DROP TABLE IF EXISTS temp_tracking_data_core_table;\n\t\tSELECT * FROM raw_telematics_data.tracking_data_core LIMIT 1;\n\t\t\n\t\tCREATE TEMP TABLE temp_tracking_data_core_table as\n SELECT *\n "..., cursorOptions=<optimized out>, boundParams=<optimized out>) at postgres.c:910
1749556806956 2025-06-10T12:00:06.956Z #12 0x000000556dcd6884 in pg_plan_queries (querytrees=querytrees@entry=0x55814891c8, query_string=0x5581479c48 "\n DROP TABLE IF EXISTS temp_tracking_data_core_table;\n\t\tSELECT * FROM raw_telematics_data.tracking_data_core LIMIT 1;\n\t\t\n\t\tCREATE TEMP TABLE temp_tracking_data_core_table as\n SELECT *\n "..., cursorOptions=2048, boundParams=boundParams@entry=0x0) at postgres.c:1002
1749556806956 2025-06-10T12:00:06.956Z #13 0x000000556de08a8c in BuildCachedPlan (plansource=plansource@entry=0x55814667c8, qlist=0x55814891c8, qlist@entry=0x0, boundParams=boundParams@entry=0x0, queryEnv=queryEnv@entry=0x0) at plancache.c:962
1749556806956 2025-06-10T12:00:06.956Z #14 0x000000556de091d8 in GetCachedPlan (plansource=plansource@entry=0x55814667c8, boundParams=0x0, owner=owner@entry=0x0, queryEnv=0x0) at plancache.c:1244
1749556806956 2025-06-10T12:00:06.956Z #15 0x000000556db64660 in _SPI_execute_plan (plan=plan@entry=0x7fe57d0530, options=options@entry=0x7fe57d0740, snapshot=snapshot@entry=0x0, crosscheck_snapshot=crosscheck_snapshot@entry=0x0, fire_triggers=fire_triggers@entry=true) at spi.c:2577
1749556806956 2025-06-10T12:00:06.956Z #16 0x000000556db64f8c in SPI_execute_extended (src=src@entry=0x5581479c48 "\n DROP TABLE IF EXISTS temp_tracking_data_core_table;\n\t\tSELECT * FROM raw_telematics_data.tracking_data_core LIMIT 1;\n\t\t\n\t\tCREATE TEMP TABLE temp_tracking_data_core_table as\n SELECT *\n "..., options=options@entry=0x7fe57d0740) at spi.c:662
1749556806956 2025-06-10T12:00:06.956Z #17 0x0000007f6fec1178 in exec_stmt_dynexecute (stmt=0x5581468a68, estate=0x7fe57d0a20) at pl_exec.c:4488
1749556806956 2025-06-10T12:00:06.956Z #18 exec_stmts (estate=estate@entry=0x7fe57d0a20, stmts=0x5581468978) at pl_exec.c:2096
1749556806956 2025-06-10T12:00:06.956Z #19 0x0000007f6fec3478 in exec_stmt_block (estate=estate@entry=0x7fe57d0a20, block=block@entry=0x558146ab30) at pl_exec.c:1943
1749556806956 2025-06-10T12:00:06.956Z #20 0x0000007f6fec3588 in exec_toplevel_block (estate=estate@entry=0x7fe57d0a20, block=0x558146ab30) at pl_exec.c:1634
1749556806956 2025-06-10T12:00:06.956Z #21 0x0000007f6fec3df0 in plpgsql_exec_function (func=func@entry=0x5581380a40, fcinfo=fcinfo@entry=0x5581459200, simple_eval_estate=simple_eval_estate@entry=0x0, simple_eval_resowner=simple_eval_resowner@entry=0x0, procedure_resowner=procedure_resowner@entry=0x0, atomic=atomic@entry=true) at pl_exec.c:623
1749556806956 2025-06-10T12:00:06.956Z #22 0x0000007f6fece564 in plpgsql_call_handler (fcinfo=0x5581459200) at pl_handler.c:277
1749556806956 2025-06-10T12:00:06.956Z #23 0x000000556db25fbc in ExecInterpExpr (state=0x55814590b8, econtext=0x5581458e88, isnull=0x7fe57d0d9f) at execExprInterp.c:740
1749556806956 2025-06-10T12:00:06.956Z #24 0x000000556db58cb4 in ExecEvalExprSwitchContext (isNull=0x7fe57d0d9f, econtext=0x5581458e88, state=<optimized out>) at ../../../src/include/executor/executor.h:356
1749556806956 2025-06-10T12:00:06.956Z #25 ExecProject (projInfo=<optimized out>) at ../../../src/include/executor/executor.h:390
1749556806956 2025-06-10T12:00:06.956Z #26 ExecResult (pstate=<optimized out>) at nodeResult.c:135
1749556806956 2025-06-10T12:00:06.956Z #27 0x000000556db29ef4 in ExecProcNode (node=0x5581458d80) at ../../../src/include/executor/executor.h:274
1749556806956 2025-06-10T12:00:06.956Z #28 ExecutePlan (dest=0x55812db358, direction=<optimized out>, numberTuples=0, sendTuples=true, operation=CMD_SELECT, queryDesc=0x55812b24d8) at execMain.c:1649
1749556806956 2025-06-10T12:00:06.956Z #29 standard_ExecutorRun (queryDesc=0x55812b24d8, direction=<optimized out>, count=0, execute_once=<optimized out>) at execMain.c:361
1749556806956 2025-06-10T12:00:06.956Z #30 0x0000007f97c74558 in pgss_ExecutorRun (queryDesc=0x55812b24d8, direction=ForwardScanDirection, count=0, execute_once=false) at pg_stat_statements.c:1025
1749556806956 2025-06-10T12:00:06.956Z #31 0x000000556dcd9894 in PortalRunSelect (portal=portal@entry=0x55813dc668, forward=forward@entry=true, count=0, count@entry=9223372036854775807, dest=dest@entry=0x55812db358) at pquery.c:922
1749556806956 2025-06-10T12:00:06.956Z #32 0x000000556dcdae08 in PortalRun (portal=portal@entry=0x55813dc668, count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=true, run_once=run_once@entry=true, dest=dest@entry=0x55812db358, altdest=altdest@entry=0x55812db358, qc=qc@entry=0x7fe57d1270) at pquery.c:766
1749556806956 2025-06-10T12:00:06.956Z #33 0x000000556dcd6c84 in exec_simple_query (query_string=query_string@entry=0x55812d99b8 "SELECT raw_telematics_data.transfer_tracking_data();") at postgres.c:1280
1749556806956 2025-06-10T12:00:06.956Z #34 0x000000556dcd85b4 in PostgresMain (dbname=<optimized out>, username=<optimized out>) at postgres.c:4777
1749556806957 2025-06-10T12:00:06.957Z #35 0x000000556dcd2c20 in BackendMain (startup_data=<optimized out>, startup_data_len=<optimized out>) at backend_startup.c:105
1749556806957 2025-06-10T12:00:06.957Z #36 0x000000556dc3add8 in postmaster_child_launch (child_type=child_type@entry=B_BACKEND, startup_data=startup_data@entry=0x7fe57d1748 "", startup_data_len=startup_data_len@entry=4, client_sock=client_sock@entry=0x7fe57d1750) at launch_backend.c:277
1749556806957 2025-06-10T12:00:06.957Z #37 0x000000556dc3e6b0 in BackendStartup (client_sock=0x7fe57d1750) at postmaster.c:3595
1749556806957 2025-06-10T12:00:06.957Z #38 ServerLoop () at postmaster.c:1676
1749556806957 2025-06-10T12:00:06.957Z #39 0x000000556dc404d8 in PostmasterMain (argc=argc@entry=3, argv=argv@entry=0x5581295400) at postmaster.c:1374
1749556806957 2025-06-10T12:00:06.957Z #40 0x000000556d955aec in main (argc=3, argv=0x5581295400) at main.c:238
To Reproduce
I'll need to get a reproduction from our customer or do some testing myself if you all think it is necessary. More than happy to do so 😄
OS:
Linux (Debian Bullseye) x86_64
pg_duckdb Version (if built from source use commit hash):
bd3c5a0fffe187358a7d66c888bf0ea2e5afb8a1
Postgres Version (if built from source use commit hash):
Neon's fork of 17.5: https://github.com/neondatabase/postgres/tree/REL_17_STABLE_neon
Hardware:
No response
Full Name:
Tristan Partin
Affiliation:
Databricks
What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.
I have not tested with any build
Did you include all relevant data sets for reproducing the issue?
No - Other reason (please specify in the issue body)
Did you include all code required to reproduce the issue?
- [ ] Yes, I have
Did you include all relevant configuration (e.g., CPU architecture, Linux distribution) to reproduce the issue?
- [x] Yes, I have
Can you check if this PR solves the issue: https://github.com/duckdb/pg_duckdb/pull/846
I'll set up a test env tomorrow. Thanks for your help! Looks good as is though.