pg_pathman icon indicating copy to clipboard operation
pg_pathman copied to clipboard

Published 20 hours ago • verified publisher icon postgrespro

When I execute a use case of pg_pathman on postgresql 12.7, a core file is generated.

Open LJXLJX1 opened this issue 4 years ago • 2 comments

Problem description

When I execute a use case of pg_pathman on postgresql 12.7, a core file is generated

core file： [dytestpg@host-10-57-68-13 corefile]$ gdb /home/dytestpg/postgres/bin/postgres /corefile/core-1625843430-postgres-18236-6

Errors： void heap_insert(Relation relation, HeapTuple tup, CommandId cid, int options, BulkInsertState bistate) { TransactionId xid = GetCurrentTransactionId(); HeapTuple heaptup; Buffer buffer; Buffer vmbuffer = InvalidBuffer; bool all_visible_cleared = false;

/* Cheap, simplistic check that the tuple matches the rel's rowtype. */
Assert(HeapTupleHeaderGetNatts(tup->t_data) <=
	   RelationGetNumberOfAttributes(relation));

Breakpoint 1, heap_insert (relation=0x7f8a3a221998, tup=0x1471e70, cid=0, options=0, bistate=0x0) at heapam.c:1883 3: * relation->rd_rel = {oid = 73801, relname = {data = "test_12", '\000' <repeats 56 times>}, relnamespace = 73728, reltype = 73803, reloftype = 0, relowner = 10, relam = 2, relfilenode = 73801, reltablespace = 0, relpages = 0, reltuples = 0, relallvisible = 0, reltoastrelid = 0, relhasindex = false, relisshared = false, relpersistence = 112 'p', relkind = 114 'r', relnatts = 2, relchecks = 1, relhasrules = false, relhastriggers = false, relhassubclass = false, relrowsecurity = false, relforcerowsecurity = false, relispopulated = true, relreplident = 100 'd', relispartition = false, relrewrite = 0, relfrozenxid = 1136, relminmxid = 1} 2: RelationGetNumberOfAttributes(relation) = 2 1: HeapTupleHeaderGetNatts(tup->t_data) = 3

The use cases that report errors are executed in sequence, and errors will be reported at the end.

/*

  • NOTE: This test behaves differenly on < 11 because planner now turns
  • Row(Const, Const) into just Const of record type, apparently since 3decd150

/ \set VERBOSITY terse SET search_path = 'public'; CREATE EXTENSION pg_pathman; CREATE SCHEMA test_updates; /

  • Test UPDATEs on a partition with different TupleDescriptor. / / create partitioned table */ CREATE TABLE test_updates.test(a FLOAT4, val INT4 NOT NULL, b FLOAT8); INSERT INTO test_updates.test SELECT i, i, i FROM generate_series(1, 100) AS i; SELECT create_range_partitions('test_updates.test', 'val', 1, 10); create_range_partitions
                  10

(1 row)

/* drop column 'a' / ALTER TABLE test_updates.test DROP COLUMN a; / append new partition */ SELECT append_range_partition('test_updates.test'); append_range_partition

test_updates.test_11 (1 row)

INSERT INTO test_updates.test_11 (val, b) VALUES (101, 10); VACUUM ANALYZE; /* tuple descs are the same */ EXPLAIN (COSTS OFF) UPDATE test_updates.test SET b = 0 WHERE val = 1; QUERY PLAN

Update on test_1 -> Seq Scan on test_1 Filter: (val = 1) (3 rows)

UPDATE test_updates.test SET b = 0 WHERE val = 1 RETURNING *, tableoid::REGCLASS; val | b | tableoid -----+---+--------------------- 1 | 0 | test_updates.test_1 (1 row)

/* tuple descs are different */ EXPLAIN (COSTS OFF) UPDATE test_updates.test SET b = 0 WHERE val = 101; QUERY PLAN

Update on test_11 -> Seq Scan on test_11 Filter: (val = 101) (3 rows)

UPDATE test_updates.test SET b = 0 WHERE val = 101 RETURNING *, tableoid::REGCLASS; val | b | tableoid -----+---+---------------------- 101 | 0 | test_updates.test_11 (1 row)

CREATE TABLE test_updates.test_dummy (val INT4); EXPLAIN (COSTS OFF) UPDATE test_updates.test SET val = val + 1 WHERE val = 101 AND val = ANY (TABLE test_updates.test_dummy) RETURNING *, tableoid::REGCLASS; QUERY PLAN

Update on test_11 -> Nested Loop Semi Join -> Seq Scan on test_11 Filter: (val = 101) -> Seq Scan on test_dummy Filter: (val = 101) (6 rows)

EXPLAIN (COSTS OFF) UPDATE test_updates.test t1 SET b = 0 FROM test_updates.test_dummy t2 WHERE t1.val = 101 AND t1.val = t2.val RETURNING t1.*, t1.tableoid::REGCLASS; QUERY PLAN

Update on test_11 t1 -> Nested Loop -> Seq Scan on test_11 t1 Filter: (val = 101) -> Seq Scan on test_dummy t2 Filter: (val = 101) (6 rows)

EXPLAIN (COSTS OFF) UPDATE test_updates.test SET b = 0 WHERE val = 101 AND test >= (100, 8) RETURNING *, tableoid::REGCLASS; QUERY PLAN

Update on test_11 -> Seq Scan on test_11 Filter: (((test_11.*)::test_updates.test >= '(100,8)'::record) AND (val = 101)) (3 rows)

/* execute this one */ UPDATE test_updates.test SET b = 0 WHERE val = 101 AND test >= (100, -1) RETURNING test; test

(101,0) (1 row)

DROP TABLE test_updates.test_dummy; /* cross-partition updates (& different tuple descs) */ TRUNCATE test_updates.test; SET pg_pathman.enable_partitionrouter = ON; SELECT , (select count() from pg_attribute where attrelid = partition) as columns FROM pathman_partition_list ORDER BY range_min::int, range_max::int; parent | partition | parttype | expr | range_min | range_max | columns -------------------+----------------------+----------+------+-----------+-----------+--------- test_updates.test | test_updates.test_1 | 2 | val | 1 | 11 | 9 test_updates.test | test_updates.test_2 | 2 | val | 11 | 21 | 9 test_updates.test | test_updates.test_3 | 2 | val | 21 | 31 | 9 test_updates.test | test_updates.test_4 | 2 | val | 31 | 41 | 9 test_updates.test | test_updates.test_5 | 2 | val | 41 | 51 | 9 test_updates.test | test_updates.test_6 | 2 | val | 51 | 61 | 9 test_updates.test | test_updates.test_7 | 2 | val | 61 | 71 | 9 test_updates.test | test_updates.test_8 | 2 | val | 71 | 81 | 9 test_updates.test | test_updates.test_9 | 2 | val | 81 | 91 | 9 test_updates.test | test_updates.test_10 | 2 | val | 91 | 101 | 9 test_updates.test | test_updates.test_11 | 2 | val | 101 | 111 | 8 (11 rows)

INSERT INTO test_updates.test VALUES (105, 105); UPDATE test_updates.test SET val = 106 WHERE val = 105 RETURNING *, tableoid::REGCLASS; val | b | tableoid -----+-----+---------------------- 106 | 105 | test_updates.test_11 (1 row)

UPDATE test_updates.test SET val = 115 WHERE val = 106 RETURNING *, tableoid::REGCLASS; server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. connection to server was lost

Environment

postgres=# SELECT * FROM pg_extension postgres-# ; oid | extname | extowner | extnamespace | extrelocatable | extversion | extconfig | extcondition -------+------------+----------+--------------+----------------+------------+---------------+-------------- 13579 | plpgsql | 10 | 11 | f | 1.0 | | 16384 | pg_pathman | 10 | 2200 | f | 1.5 | {16386,16397} | {"",""} (2 rows)

postgres=# SELECT version(); version

PostgreSQL 12.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28), 64-bit (1 row)

LJXLJX1 avatar Jul 12 '21 08:07 LJXLJX1

问题描述

当我在 postgresql 12.7 上执行 pg_pathman 的用例时，会生成一个核心文件

核心文件： [dytestpg@host-10-57-68-13 corefile]$ gdb /home/dytestpg/postgres/bin/postgres /corefile/core-1625843430-postgres-18236-6

错误： void heap_insert(Relation relationship, HeapTuple tup, CommandId cid, int options, BulkInsertState bistate) { TransactionId xid = GetCurrentTransactionId(); HeapTuple heapup; 缓冲缓冲区； 缓冲区 vmbuffer = InvalidBuffer; bool all_visible_cleared = false;

/* Cheap, simplistic check that the tuple matches the rel's rowtype. */
Assert(HeapTupleHeaderGetNatts(tup->t_data) <=
	   RelationGetNumberOfAttributes(relation));

断点 1, heap_insert (relation=0x7f8a3a221998, tup=0x1471e70, cid=0, options=0, bistate=0x0) at heapam.c:1883 3: * relation->rd_rel = {oid = 73801, relname = {data = " test_12", '\000' <重复 56 次>}, relnamespace = 73728, reltype = 73803, reloftype = 0, relowner = 10, relam = 2, relfilenode = 73801, reltablespace = 0, relpages = 0, reltuples = 0, relallvisible = 0，reltoastrelid = 0，relhasindex = false，relisshared = false，relpersistence = 112 'p'， relkind = 114 'r'，relnatts = 2，relchecks = 1，relhasrules = false，relhastriggers = false，relhassubclass = false， relrowsecurity = false，relforcerowsecurity = false， relispopulated = true，relreplident = 100 'd'，relispartition = false，relrewrite = 0，relfrozenxid = 1136，relminmxid = 1} 2: RelationGetNumberOfAttributes(relation) = 2 1: HeapTupleHeaderGetNatts(tup->t_data) = 3

报错的用例依次执行，最后报错。

/*

  • 注意：此测试在 < 11 时表现不同，因为规划器现在转向
  • Row(Const, Const) 变成记录类型的 Const，显然是从 3decd150

/ \set VERBOSITY terse SET search_path = 'public'; 创建扩展 pg_pathman; 创建模式测试更新； /

  • 在具有不同 TupleDescriptor 的分区上测试更新。 _/ /_创建分区表* / CREATE TABLE test_updates.test（一个FLOAT4，VAL INT4 NOT NULL，B是float8）; INSERT INTO test_updates.test SELECT i, i, i FROM generate_series(1, 100) AS i; 选择 create_range_partitions('test_updates.test', 'val', 1, 10); 创建范围分区
                  10

(1 行)

/* 删除列 'a' _/ **ALTER TABLE test_updates.test DROP COLUMN a; ** /_追加新分区 */

SELECT append_range_partition('test_updates.test'); append_range_partition test_updates.test_11 （1 行）

插入到 test_updates.test_11 (val, b) VALUES (101, 10);

真空分析； /* tuple desc 相同 */ EXPLAIN (COSTS OFF) UPDATE test_updates.test SET b = 0 WHERE val = 1; 查询计划 更新 test_1 -> 对 test_1 过滤器进行Seq 扫描：（val = 1） （3 行）

更新 test_updates.test SET b = 0 WHERE val = 1 RETURNING *, tableoid::REGCLASS; 值 | 乙 | tableoid -----+---+--------------------- 1 | 0 | test_updates.test_1 （1 行）

/* tuple desc 不同 */

EXPLAIN (COSTS OFF) UPDATE test_updates.test SET b = 0 WHERE val = 101; 查询计划 更新 test_11 -> 对 test_11 过滤器进行Seq 扫描：（val = 101） （3 行）

更新 test_updates.test SET b = 0 WHERE val = 101 RETURNING *, tableoid::REGCLASS; 值 | 乙 | tableoid -----+---+---------- 101 | 0 | test_updates.test_11 （1 行）

创建表 test_updates.test_dummy (val INT4);

解释（成本关闭）更新 test_updates.test SET val = val + 1 WHERE val = 101 AND val = ANY (TABLE test_updates.test_dummy) RETURNING *, tableoid::REGCLASS; 查询计划 更新 test_11 -> 嵌套循环半连接 -> test_11 过滤器上的 Seq 扫描：（val = 101） -> test_dummy 过滤器上的 Seq 扫描：（val = 101） （6 行）

解释（成本关闭）更新 test_updates.test t1 SET b = 0

FROM test_updates.test_dummy t2 WHERE t1.val = 101 AND t1.val = t2.val RETURNING t1.*, t1.tableoid::REGCLASS; 查询计划 在 test_11 t1 上更新 -> 嵌套循环 -> 在 test_11 t1 过滤器上进行 Seq 扫描：(val = 101) -> 在 test_dummy t2 过滤器上进行 Seq 扫描：(val = 101) （6 行）

解释（成本关闭）更新 test_updates.test SET b = 0

WHERE val = 101 AND test >= (100, 8) RETURNING , tableoid::REGCLASS; 查询计划 更新 test_11 -> 对 test_11 过滤器进行Seq 扫描：(((test_11.)::test_updates.test >= '(100,8)'::record) AND (val = 101)) (3 行)

/* 执行这个 */

UPDATE test_updates.test SET b = 0 WHERE val = 101 AND test >= (100, -1) RETURNING test; 测试 (101,0) (1 行)

删除表 test_updates.test_dummy; /* 跨分区更新（和不同的元组描述） */ TRUNCATE test_updates.test; 设置 pg_pathman.enable_partitionrouter = ON; SELECT , (select count( ) from pg_attribute where attrelid = partition) 作为列 FROM pathman_partition_list ORDER BY range_min::int, range_max::int; 家长 | 隔断| 零件类型 | 表达式 | range_min | range_max | 列 -------------------+------------------------+------ ----+------+------------+------------+--------- test_updates.test | test_updates.test_1 | 2 | 值 | 1 | 11 | 9 test_updates.test | test_updates.test_2 | 2 | 值 | 11 | 21 | 9 test_updates.test | test_updates.test_3 | 2 | 值 | 21 | 31 | 9 test_updates.test | test_updates.test_4 | 2 | 值 | 31 | 41 | 9 test_updates.test | test_updates.test_5 | 2 | 值 | 41 | 51 | 9 test_updates.test | test_updates.test_6 | 2 | 值 | 51 | 61 | 9 test_updates.test | test_updates.test_7 | 2 | 值 | 61 | 71 | 9 test_updates.test | test_updates.test_8 | 2 | 值 | 71 | 81 | 9 test_updates.test | test_updates.test_9 | 2 | 值 | 81 | 91 | 9 test_updates.test | test_updates.test_10 | 2 | 值 | 91 | 101 | 9 test_updates.test | test_updates.test_11 | 2 | 值 | 101 | 111 | 8 个 （11 行）

插入到 test_updates.test 值（105、105）； 更新 test_updates.test SET val = 106 WHERE val = 105 RETURNING *, tableoid::REGCLASS; 值 | 乙 | 表格 -----+-----+---------- 106 | 105 | test_updates.test_11 （1 行）

更新 test_updates.test SET val = 115 WHERE val = 106 RETURNING *, tableoid::REGCLASS; 服务器意外关闭连接 这可能意味着服务器 在处理请求之前或期间异常终止。 与服务器的连接丢失

环境

postgres=# SELECT * FROM pg_extension postgres-# ; 类 | 分机名 | 所有者 | 扩展名空间 | 可移动| 外向版 | 扩展配置 | 扩展条件 --------+------------+----------+--------------+-- --------------+------------+----------------+------ -------- 13579 | plpgsql | 10 | 11 | f | 1.0 | | 16384 | pg_pathman | 10 | 2200 | f | 1.5 | {16386,16397} | {"",""} (2 行)

postgres=#选择版本（）；

版本 x86_64-pc-linux-gnu 上的 PostgreSQL 12.7，由 gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28) 编译，64 位 （1 行）

i got same problem on postgresql 13.3，how you fixed it？

liangkdm avatar Jan 11 '22 06:01 liangkdm

i have fixed this problem by using master branch. do not use version release 1.5.12

liangkdm avatar Jan 13 '22 01:01 liangkdm