yugabyte-db icon indicating copy to clipboard operation
yugabyte-db copied to clipboard

[YSQL] COPY FROM command exiting with error -ERROR: IO error: Errors occurred while reaching out to the tablet servers:

Open priyanshi-yb opened this issue 1 year ago • 1 comments

Jira Link: DB-3594

Description

Trying to Import a data file which has 350M rows into a table using the Copy from command and getting the below error after some hours.

Schema of the table -

                  Table "public.accounts"
      Column      |  Type  | Collation | Nullable | Default 
------------------+--------+-----------+----------+---------
 block            | bigint |           | not null | 
 address          | text   |           | not null | 
 dc_balance       | bigint |           | not null | 0
 dc_nonce         | bigint |           | not null | 0
 security_balance | bigint |           | not null | 0
 security_nonce   | bigint |           | not null | 0
 balance          | bigint |           | not null | 0
 nonce            | bigint |           | not null | 0
 staked_balance   | bigint |           |          | 
Indexes:
    "accounts_pkey" PRIMARY KEY, lsm (block HASH, address ASC)
    "account_address_idx" lsm (address HASH)
    "account_block_idx" lsm (block HASH)
Triggers:
    account_insert AFTER INSERT ON accounts FOR EACH ROW EXECUTE PROCEDURE account_inventory_update()

Error -

etl2_final_tables=# \copy accounts from '/data/small-data/accounts_350m_data.sql' WITH (ROWS_PER_TRANSACTION 20000);
ERROR:  IO error: Errors occurred while reaching out to the tablet servers: . Errors from tablet servers: [Operation failed. Try again (yb/tablet/transaction_participant.cc:285): Unknown transaction, could be recently aborted: c724084d-efe5-4cf9-81df-b48325229577 (pgsql error 40001), Query error (yb/tserver/tablet_service.cc:1734): The catalog snapshot used for this transaction has been invalidated.]
CONTEXT:  Catalog Version Mismatch: A DDL occurred while processing this query. Try again.
Time: 9884572.283 ms (02:44:44.572)

These were some session variables which were set -

SET client_encoding TO 'UTF8';
SET session_replication_role TO replica;
SET yb_enable_upsert_mode to true;

YugabyteDB Version - 11.2-YB-2.14.2.0-b0 Data being ingested is of 350M rows.

priyanshi-yb avatar Sep 21 '22 12:09 priyanshi-yb

From commit c6b90bbb173be7011498fe09d0e63018fecf6f85

  1. Disable batching if the table contains non-FK trigger as per the following change.

which seems to apply to the schema given above.

tedyu avatar Sep 21 '22 15:09 tedyu