yugabyte-db
yugabyte-db copied to clipboard
[YSQL] COPY FROM command exiting with error -ERROR: IO error: Errors occurred while reaching out to the tablet servers:
Jira Link: DB-3594
Description
Trying to Import a data file which has 350M rows into a table using the Copy from
command and getting the below error after some hours.
Schema of the table -
Table "public.accounts"
Column | Type | Collation | Nullable | Default
------------------+--------+-----------+----------+---------
block | bigint | | not null |
address | text | | not null |
dc_balance | bigint | | not null | 0
dc_nonce | bigint | | not null | 0
security_balance | bigint | | not null | 0
security_nonce | bigint | | not null | 0
balance | bigint | | not null | 0
nonce | bigint | | not null | 0
staked_balance | bigint | | |
Indexes:
"accounts_pkey" PRIMARY KEY, lsm (block HASH, address ASC)
"account_address_idx" lsm (address HASH)
"account_block_idx" lsm (block HASH)
Triggers:
account_insert AFTER INSERT ON accounts FOR EACH ROW EXECUTE PROCEDURE account_inventory_update()
Error -
etl2_final_tables=# \copy accounts from '/data/small-data/accounts_350m_data.sql' WITH (ROWS_PER_TRANSACTION 20000);
ERROR: IO error: Errors occurred while reaching out to the tablet servers: . Errors from tablet servers: [Operation failed. Try again (yb/tablet/transaction_participant.cc:285): Unknown transaction, could be recently aborted: c724084d-efe5-4cf9-81df-b48325229577 (pgsql error 40001), Query error (yb/tserver/tablet_service.cc:1734): The catalog snapshot used for this transaction has been invalidated.]
CONTEXT: Catalog Version Mismatch: A DDL occurred while processing this query. Try again.
Time: 9884572.283 ms (02:44:44.572)
These were some session variables which were set -
SET client_encoding TO 'UTF8';
SET session_replication_role TO replica;
SET yb_enable_upsert_mode to true;
YugabyteDB Version - 11.2-YB-2.14.2.0-b0
Data being ingested is of 350M rows.
From commit c6b90bbb173be7011498fe09d0e63018fecf6f85
- Disable batching if the table contains non-FK trigger as per the following change.
which seems to apply to the schema given above.