Segmentation fault, use table partitioning。

Open zhang-wenchao opened this issue 5 months ago • 1 comments

What happens?

Segmentation fault

2025-06-27 13:54:00.222 CEST [2636294] LOG:  server process (PID 2645025) was terminated by signal 11: Segmentation fault
2025-06-27 13:54:00.222 CEST [2636294] DETAIL:  Failed process was running: SELECT 'data.events_2015' as partition_name, COUNT(*) as row_count FROM data.events_2015
2025-06-27 13:54:00.223 CEST [2636294] LOG:  terminating any other active server processes
2025-06-27 13:54:00.314 CEST [2636294] LOG:  all server processes terminated; reinitializing
2025-06-27 13:54:01.421 CEST [2645413] LOG:  database system was interrupted; last known up at 2025-06-27 13:46:12 CEST
2025-06-27 13:54:01.470 CEST [2645413] LOG:  database system was not properly shut down; automatic recovery in progress
2025-06-27 13:54:01.472 CEST [2645413] LOG:  redo starts at 5F/E2B09818
2025-06-27 13:54:01.472 CEST [2645413] LOG:  invalid record length at 5F/E2B09970: expected at least 24, got 0
2025-06-27 13:54:01.472 CEST [2645413] LOG:  redo done at 5F/E2B09938 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s

To Reproduce

CREATE TABLE "data"."events"
(
    "id"          BIGINT                   NOT NULL,
    "actor_id"    BIGINT                   NOT NULL,
    "actor_login" TEXT                     NOT NULL,
    "repo_id"     BIGINT,
    "repo_name"   TEXT,
    "org_id"      BIGINT,
    "org_login"   TEXT,
    "event_type"  TEXT                     NOT NULL,
    "payload"     JSON                     NOT NULL DEFAULT '{}',
    "body"        TEXT,
    "abnormal"    INTEGER                  NOT NULL DEFAULT 0,
    "created_at"  TIMESTAMP WITH TIME ZONE NOT NULL,
    PRIMARY KEY ("id", "created_at")
) PARTITION BY RANGE (created_at);

CREATE TABLE IF NOT EXISTS data.events_2015 PARTITION OF data.events FOR VALUES FROM ('2015-01-01 00:00:00+00') TO ('2016-01-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2016 PARTITION OF data.events FOR VALUES FROM ('2016-01-01 00:00:00+00') TO ('2017-01-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2017 PARTITION OF data.events FOR VALUES FROM ('2017-01-01 00:00:00+00') TO ('2018-01-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2018 PARTITION OF data.events FOR VALUES FROM ('2018-01-01 00:00:00+00') TO ('2019-01-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2019 PARTITION OF data.events FOR VALUES FROM ('2019-01-01 00:00:00+00') TO ('2020-01-01 00:00:00+00');

CREATE TABLE IF NOT EXISTS data.events_2020_q1 PARTITION OF data.events FOR VALUES FROM ('2020-01-01 00:00:00+00') TO ('2020-04-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2020_q2 PARTITION OF data.events FOR VALUES FROM ('2020-04-01 00:00:00+00') TO ('2020-07-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2020_q3 PARTITION OF data.events FOR VALUES FROM ('2020-07-01 00:00:00+00') TO ('2020-10-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2020_q4 PARTITION OF data.events FOR VALUES FROM ('2020-10-01 00:00:00+00') TO ('2021-01-01 00:00:00+00');

CREATE TABLE IF NOT EXISTS data.events_2021_q1 PARTITION OF data.events FOR VALUES FROM ('2021-01-01 00:00:00+00') TO ('2021-04-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2021_q2 PARTITION OF data.events FOR VALUES FROM ('2021-04-01 00:00:00+00') TO ('2021-07-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2021_q3 PARTITION OF data.events FOR VALUES FROM ('2021-07-01 00:00:00+00') TO ('2021-10-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2021_q4 PARTITION OF data.events FOR VALUES FROM ('2021-10-01 00:00:00+00') TO ('2022-01-01 00:00:00+00');

CREATE TABLE IF NOT EXISTS data.events_2022_q1 PARTITION OF data.events FOR VALUES FROM ('2022-01-01 00:00:00+00') TO ('2022-04-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2022_q2 PARTITION OF data.events FOR VALUES FROM ('2022-04-01 00:00:00+00') TO ('2022-07-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2022_q3 PARTITION OF data.events FOR VALUES FROM ('2022-07-01 00:00:00+00') TO ('2022-10-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2022_q4 PARTITION OF data.events FOR VALUES FROM ('2022-10-01 00:00:00+00') TO ('2023-01-01 00:00:00+00');

CREATE TABLE IF NOT EXISTS data.events_2023_q1 PARTITION OF data.events FOR VALUES FROM ('2023-01-01 00:00:00+00') TO ('2023-04-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2023_q2 PARTITION OF data.events FOR VALUES FROM ('2023-04-01 00:00:00+00') TO ('2023-07-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2023_q3 PARTITION OF data.events FOR VALUES FROM ('2023-07-01 00:00:00+00') TO ('2023-10-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2023_q4 PARTITION OF data.events FOR VALUES FROM ('2023-10-01 00:00:00+00') TO ('2024-01-01 00:00:00+00');

CREATE TABLE IF NOT EXISTS data.events_2024_q1 PARTITION OF data.events FOR VALUES FROM ('2024-01-01 00:00:00+00') TO ('2024-04-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2024_q2 PARTITION OF data.events FOR VALUES FROM ('2024-04-01 00:00:00+00') TO ('2024-07-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2024_q3 PARTITION OF data.events FOR VALUES FROM ('2024-07-01 00:00:00+00') TO ('2024-10-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2024_q4 PARTITION OF data.events FOR VALUES FROM ('2024-10-01 00:00:00+00') TO ('2025-01-01 00:00:00+00');

CREATE TABLE IF NOT EXISTS data.events_2025_01 PARTITION OF data.events FOR VALUES FROM ('2025-01-01 00:00:00+00') TO ('2025-02-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2025_02 PARTITION OF data.events FOR VALUES FROM ('2025-02-01 00:00:00+00') TO ('2025-03-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2025_03 PARTITION OF data.events FOR VALUES FROM ('2025-03-01 00:00:00+00') TO ('2025-04-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2025_04 PARTITION OF data.events FOR VALUES FROM ('2025-04-01 00:00:00+00') TO ('2025-05-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2025_05 PARTITION OF data.events FOR VALUES FROM ('2025-05-01 00:00:00+00') TO ('2025-06-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2025_06 PARTITION OF data.events FOR VALUES FROM ('2025-06-01 00:00:00+00') TO ('2025-07-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2025_07 PARTITION OF data.events FOR VALUES FROM ('2025-07-01 00:00:00+00') TO ('2025-08-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2025_08 PARTITION OF data.events FOR VALUES FROM ('2025-08-01 00:00:00+00') TO ('2025-09-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2025_09 PARTITION OF data.events FOR VALUES FROM ('2025-09-01 00:00:00+00') TO ('2025-10-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2025_10 PARTITION OF data.events FOR VALUES FROM ('2025-10-01 00:00:00+00') TO ('2025-11-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2025_11 PARTITION OF data.events FOR VALUES FROM ('2025-11-01 00:00:00+00') TO ('2025-12-01 00:00:00+00');
CREATE TABLE IF NOT EXISTS data.events_2025_12 PARTITION OF data.events FOR VALUES FROM ('2025-12-01 00:00:00+00') TO ('2026-01-01 00:00:00+00');

Write more than 100 million records.

SET duckdb.force_execution = true;

explain SELECT 'data.events' as partition_name, COUNT(*) as row_count FROM data.events;

Custom Scan (DuckDBScan)  (cost=0.00..0.00 rows=0 width=0)
  DuckDB Execution Plan: 
""
┌───────────────────────────┐
│         PROJECTION        │
│    ────────────────────   │
│       partition_name      │
│         row_count         │
│                           │
│          ~1 Rows          │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│    UNGROUPED_AGGREGATE    │
│    ────────────────────   │
│        Aggregates:        │
│        count_star()       │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│       POSTGRES_SCAN       │
│    ────────────────────   │
│       Table: events       │
│                           │
│          ~0 Rows          │
└───────────────────────────┘
""
""

SELECT 'data.events' as partition_name, COUNT(*) as row_count FROM data.events;

2025-06-27 13:54:00.222 CEST [2636294] LOG:  server process (PID 2645025) was terminated by signal 11: Segmentation fault
2025-06-27 13:54:00.222 CEST [2636294] DETAIL:  Failed process was running: SELECT 'data.events_2015' as partition_name, COUNT(*) as row_count FROM data.events_2015
2025-06-27 13:54:00.223 CEST [2636294] LOG:  terminating any other active server processes
2025-06-27 13:54:00.314 CEST [2636294] LOG:  all server processes terminated; reinitializing
2025-06-27 13:54:01.421 CEST [2645413] LOG:  database system was interrupted; last known up at 2025-06-27 13:46:12 CEST
2025-06-27 13:54:01.470 CEST [2645413] LOG:  database system was not properly shut down; automatic recovery in progress
2025-06-27 13:54:01.472 CEST [2645413] LOG:  redo starts at 5F/E2B09818
2025-06-27 13:54:01.472 CEST [2645413] LOG:  invalid record length at 5F/E2B09970: expected at least 24, got 0
2025-06-27 13:54:01.472 CEST [2645413] LOG:  redo done at 5F/E2B09938 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s

OS:

Linux Debian

pg_duckdb Version (if built from source use commit hash):

0.3.1

Postgres Version (if built from source use commit hash):

Hardware:

No response

Full Name:

Zhang WenChao

Affiliation:

Personal

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a stable release

Did you include all relevant data sets for reproducing the issue?

No - Other reason (please specify in the issue body)

Did you include all code required to reproduce the issue?

[x] Yes, I have

Did you include all relevant configuration (e.g., CPU architecture, Linux distribution) to reproduce the issue?

[x] Yes, I have

Jun 27 '25 12:06 zhang-wenchao

Can you try with the latest commit from the main branch? We fixed a lot of crashes since 3.1.

Jun 28 '25 20:06 JelteF