taosdump 3.2.3.0 import stuck while using multi threads

Open Whitroom opened this issue 1 year ago • 0 comments

Bug描述 TDEngine3.2.3.0对数据进行taosdump导出后，删除 /var/lib/taos/** 重启taosd后使用taosdump导入卡住不动

如何重现 使用systemctl进行进程管理

taosdump -uroot --database=ess_cloud -o td-bak
systemctl stop taosd
systemctl stop taosadapter
rm -rf /var/lib/taos/**
systemctl start taosd
systemctl start taosadapter
# 创建demo数据库
nohup taosdump -uroot --database=ess_cloud -i td-bak -g &

期待结果 可以成功导入

日志

DEBG: Client info: 3.2.3.0, major version: 3
==============================
========== arguments config =========
version: 3.2.3.0
gitinfo: 3bb61cc
buildInfo: Built Linux-x64 at 2024-02-29 18:01:09 +0800
 host: (null)
user: root
port: 0
outpath: 
inpath: td-bak
resultFile: ./dump_result.txt
all_databases: false
databases: true
databasesSeq: ess_cloud
schemaonly: false
with_property: true
avro codec: snappy
data_batch: 16383
thread_num: 128
allow_sys: false
escape_char: true
loose_mode: false
isDumpIn: true
arg_list_len: 0
buildInfo: Built Linux-x64 at 2024-02-29 18:01:09 +0800
 start time: 2024-06-26 11:34:09
INFO: dumpInWithDbPath(), dump in from td-bak ...
DEBG: Success Open input file: td-bak/dbs.sql
DEBG: dumpInDbs() LN10525, dump in data minor version is: 2
INFO: dumpInAvroWorkThreads() dump in avro-tbtags files ...
DEBG: 0 .avro-tbtags files found!
DEBG: No .avro-tbtags file found in td-bak
INFO: dumpInAvroWorkThreads() dump in avro-ntb files ...
DEBG: 0 .avro-ntb files found!
DEBG: No .avro-ntb file found in td-bak
INFO: dumpInAvroWorkThreads() dump in avro files ...
DEBG: 0 .avro files found!
DEBG: No .avro file found in td-bak
DEBG: dumpIn() LN10643, will dump from td-bak/taosdump.3438732572160
NFO: dumpInWithDbPath(), dump in from td-bak/taosdump.3438732572160 ...
DEBG: Success Open input file: td-bak/taosdump.3438732572160/dbs.sql
DEBG: dumpInDbs() LN10525, dump in data minor version is: 2
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE DATABASE IF NOT EXISTS ess_cloud REPLICA 1   DURATION 14400m KEEP 5256000m,5256000m,5256000m     PRECISION 'ms'   MINROWS 100 MAXROWS 4096 COMP 2 ;
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`aircool_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`enable_state` tinyint,`online_state` tinyint,`fault_state` 
bigint,`work_state` tinyint,`indoor_fan_state` tinyint,`outdoor_fan_state` tinyint,`compr_state` tinyint,`heat_state` tinyint,`defrost_temp` float,`condense_temp` float,`outdoor_temp` float,`indoor_temp` float,`indo
or_humi` float,`air_out_temp` float,`cool_temp` float,`cool_dif` float,`heat_temp` float,`heat_dif` float,`high_temp` float,`low_temp` float,`ctrl_humi` float,`humi_dif` float,`high_humi` float) TAGS(`air_code` bina
ry(32),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`liquidcool_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`enable_state` tinyint,`online_state` tinyint,`run_state`
 tinyint,`compr_state` tinyint,`heat_state` tinyint,`self_loop_state` tinyint,`pump_speed` float,`outdoor_temp` float,`inlet_water_temp` float,`outlet_water_temp` float,`inlet_water_press` float,`outlet_water_press`
 float,`fan_output` float,`fault_level` tinyint,`state_info` binary(64),`fault_info` binary(64),`on_off` tinyint,`set_run_mode` tinyint,`set_temp` float,`set_diff_mode` tinyint,`cool_start_dif` float,`cool_stop_dif`
 float,`heat_start_dif` float,`heat_stop_dif` float) TAGS(`liquid_code` binary(32),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`event_data`(`ts` timestamp,`event_level` nchar(16),`senior_type` nchar(16),`junior_type` nchar(16),`event_desc` nchar(256),`collect_time
` nchar(32),`write_time` nchar(32),`event_id` int,`device_type` nchar(32),`device_manufactor` nchar(32),`device_id` nchar(128),`event_status` int,`event_data_detail` nchar(256),`box_code` nchar(16),`bms_code` nchar(
16)) TAGS(`station_code` nchar(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`device_pcs_overview`(`ts` timestamp,`work_state` tinyint) TAGS(`pcs_code` binary(32),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`fenggu_data`(`ts` timestamp,`down_id` nchar(32),`down_time` nchar(32),`down_status` int,`down_user` nchar(32),`down_user_name` nchar(32)
,`down_user_phone` nchar(32),`down_data` nchar(4096)) TAGS(`station_code` nchar(32));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`fire_host_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`enable_state` tinyint,`online_state` tinyint,`alarm_state
` tinyint,`fault_state` tinyint,`spary_state` tinyint) TAGS(`fire_host_code` binary(32),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`cluster_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`soc` float,`soh` float,`voltage` double,`current` float,`le
ft_energy` double,`used_energy` double,`input_ah` double,`output_ah` double,`warn_state` bigint,`prt_state` bigint,`input_watt` double,`output_watt` double,`day_input_watt` double,`day_output_watt` double,`vmin` flo
at,`vmin_bmu_idx` smallint,`vmin_cell_idx` smallint,`vmax` float,`vmax_bmu_idx` smallint,`vmax_cell_idx` smallint,`tmin` float,`tmin_bmu_idx` smallint,`tmin_cell_idx` smallint,`tmax` float,`tmax_bmu_idx` smallint,`t
max_cell_idx` smallint,`vmin_sec` float,`vmin_sec_bmu_idx` smallint,`vmin_sec_cell_idx` smallint,`vmax_sec` float,`vmax_sec_bmu_idx` smallint,`vmax_sec_cell_idx` smallint,`tmin_sec` float,`tmin_sec_bmu_idx` smallint
,`tmin_sec_cell_idx` smallint,`tmax_sec` float,`tmax_sec_bmu_idx` smallint,`tmax_sec_cell_idx` smallint,`bmu_online_state` bigint,`run_state` bigint,`bc_prt_act_state` bigint,`brk_fault_state` bigint,`sa_online_stat
e` tinyint,`bc_self_chk_state` tinyint,`bc_fault_state` tinyint,`balance_state` bigint,`balance_en` bigint,`brk_state` bigint,`bc_sys_state` tinyint,`bmu_volt_sum` double,`power` double,`bcms_online_state` bigint,`b
cu_online_state` bigint,`main_breaker_state` tinyint,`balance_mode` tinyint,`balance_manual_mode_en` tinyint,`fan_state` int,`fan_fault` int,`limit_power_state` bigint,`limit_chg_power` double,`limit_dchg_power` dou
ble,`bmu_avg_vol` float,`cell_avg_vol` float,`cell_avg_temp` float,`vol_dif_max` float,`temp_dif_max` float,`soe` float,`sop` float,`sof` float,`pbus_resistor` float,`nbus_resistor` float,`correct` tinyint) TAGS(`cl
uster_code` binary(32),`bms_code` binary(16),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`dehumidity_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`enable_state` tinyint,`online_state` tinyint,`env_temp` 
float,`cool_temp` float,`env_humi` float,`ctrl_temp` float,`ctrl_humi` float,`humi_dif` float,`fan_state` tinyint,`cool_state` tinyint,`heat_state` tinyint,`run_mode` tinyint,`fan_fault` tinyint,`cool_fault` tinyint
,`sensor_fault` tinyint,`over_humi_alarm` tinyint,`over_temp_alarm` tinyint) TAGS(`dehumidity_code` binary(32),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`station_overview`(`ts` timestamp,`write_time` timestamp,`upper_limit_soc` float,`lower_limit_soc` float,`total_power` double,`dischargab
le_energy` double,`chargable_energy` double,`today_discharged_energy` double,`today_charged_energy` double,`max_temp` float,`min_temp` float,`max_soc` float,`min_soc` float,`station_soc` float,`poc_power` double,`po
c_on_grid_energy` double,`poc_off_grid_energy` double,`ess_power` double,`load_power` double,`discharged_energy` double,`charged_energy` double,`rated_capacity` double,`rated_power` double,`max_temp_location` binary
(32),`min_temp_location` binary(32),`max_soc_location` binary(32),`min_soc_location` binary(32),`device_type` binary(16),`correct` tinyint) TAGS(`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`leak_detector_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`enable_state` tinyint,`online_state` tinyint,`leak_st
ate` int) TAGS(`leak_detector_code` binary(32),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`pcs_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`online_state` tinyint,`ua` double,`ub` double,`uc` double,`ia` 
double,`ib` double,`ic` double,`grid_freq` float,`active_power` double,`reactive_power` double,`apparent_power` double,`power_factor` double,`udc` double,`idc` double,`pdc` double,`igbt` float,`igbta` float,`igbtb` 
float,`igbtc` float,`expect_active_power` double,`expect_reactive_power` double,`work_state` tinyint,`state_info` binary(64),`fault_info` binary(64),`ctrl_active_power` double,`ctrl_reactive_power` double,`ctrl_on_o
ff` tinyint,`status` tinyint) TAGS(`pcs_code` binary(32),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`meter_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`online_state` tinyint,`ua` double,`ub` double,`uc` double,`ia
` double,`ib` double,`ic` double,`grid_freq` float,`active_power` double,`reactive_power` double,`apparent_power` double,`power_factor` double,`combined_active_total` double,`combined_active_tip` double,`combined_ac
tive_peak` double,`combined_active_flat` double,`combined_active_valley` double,`positive_active_total` double,`positive_active_tip` double,`positive_active_peak` double,`positive_active_flat` double,`positive_activ
e_valley` double,`reverse_active_total` double,`reverse_active_tip` double,`reverse_active_peak` double,`reverse_active_flat` double,`reverse_active_valley` double,`combined_reactive_total` double,`combined_reactive
_tip` double,`combined_reactive_peak` double,`combined_reactive_flat` double,`combined_reactive_valley` double,`positive_reactive_total` double,`positive_reactive_tip` double,`positive_reactive_peak` double,`positiv
e_reactive_flat` double,`positive_reactive_valley` double,`reverse_reactive_total` double,`reverse_reactive_tip` double,`reverse_reactive_peak` double,`reverse_reactive_flat` double,`reverse_reactive_valley` double)
 TAGS(`meter_code` binary(32),`station_code` binary(8),`meter_type` binary(32));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`strategy_execute`(`ts` timestamp,`execute_time` timestamp,`target_power` double,`active_power` double,`load_power` double,`work_state` t
inyint,`soc` float) TAGS(`station_code` binary(8),`data_month` int);
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`energy_statistics`(`ts` timestamp,`discharged_energy` double,`charged_energy` double,`write_time` timestamp) TAGS(`data_tag` binary(32),
`station_id` int,`cycle_type` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`bmu_data`(`ts` timestamp,`order_no` int,`report_time` timestamp,`write_time` timestamp,`soc` float,`soh` float,`voltage` float,`current`
 float,`bcurrent` float,`online_state` tinyint,`temp_data` binary(500),`volt_data` binary(500),`balance_no` int,`fan_ctrl_pwm` int,`fan_ctrl_state` bigint,`balance_state` bigint,`cell_avg_vol` float,`cell_avg_temp` 
float,`vmax` float,`vmin` float,`vmax_idx` smallint,`vmin_idx` smallint,`tmax` float,`tmin` float,`tmax_idx` smallint,`tmin_idx` smallint,`bmu_balances` binary(512)) TAGS(`bmu_code` binary(32),`cluster_code` binary(
32),`bms_code` binary(16),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`lc_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`online_state` tinyint,`lc_run` tinyint,`lc_fault` tinyint,`main_
breaker_state` tinyint,`main_breaker_fault` tinyint,`bms_state` tinyint,`sys_run_state` bigint,`active_power` float,`reactive_power` float,`power_dc` float,`able_max_chg_power` float,`able_max_dchg_power` float,`rat
ed_power` float,`rated_capacity` float,`power_mode` tinyint,`passive_power` float) TAGS(`lc_code` binary(32),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`bms_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`vmax` double,`vmax_bc_idx` smallint,`vmax_bmu_idx` smallint,`vm
ax_cell_idx` smallint,`vmin` double,`vmin_bc_idx` smallint,`vmin_bmu_idx` smallint,`vmin_cell_idx` smallint,`tmax` float,`tmax_bc_idx` smallint,`tmax_bmu_idx` smallint,`tmax_cell_idx` smallint,`tmin` float,`tmin_bc_
idx` smallint,`tmin_bmu_idx` smallint,`tmin_cell_idx` smallint,`current` double,`voltage` double,`power` double,`soc` float,`soh` float,`volt_avg` double,`temp_avg` double,`cur_avg` double,`warn_state` bigint,`prt_s
tate` bigint,`act_state` bigint,`pwr_limit_state` bigint,`sys_state` bigint,`balance_enable_state` bigint,`bc_online_flg` bigint,`left_energy` double,`used_energy` double,`input_energy` double,`output_energy` double
,`input_ah` double,`output_ah` double,`calc_ba_soc` bigint,`show_ba_cal_flag` bigint,`show_ba_soc` bigint,`total_brk_state` bigint,`bc_brk_map_state` bigint,`bc_brk_fault_state` bigint,`chg_pwr_limit` double,`dis_ch
g_pwr_limit` double,`correct` tinyint) TAGS(`bms_code` binary(16),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`device_battery_overview`(`ts` timestamp,`discharged_energy` double,`charged_energy` double,`soc` float) TAGS(`box_code` binary(16),`stat
ion_code` binary(8),`box_type` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`event_msg`(`ts` timestamp,`event_time` timestamp,`collect_time` timestamp,`report_time` timestamp,`event_name` nchar(128),`en_name` bina
ry(256),`dev_num` smallint,`code` int,`senior_type` int,`junior_type` int,`level` tinyint,`event_type` tinyint,`happen_time` timestamp,`disappear_time` timestamp,`event_desc` nchar(512),`en_desc` binary(512),`event_
state` tinyint,`location` binary(512),`params` binary(64),`cap_num` smallint,`device_type` binary(16),`box_code` binary(16),`bms_code` binary(16),`rpt_status` int,`act_flg` int,`confirm` tinyint,`event_bits` tinyint
) TAGS(`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`fire_detector_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`enable_state` tinyint,`online_state` tinyint,`co` flo
at,`voc` float,`temp` float,`smoke` float) TAGS(`fire_detector_code` binary(32),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`glc_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`cfg_lc_sum` smallint) TAGS(`glc_code` binary(32),`station_code`
 binary(8));
INFO: dumpInAvroWorkThreads() dump in avro-tbtags files ...
DEBG: 12 .avro-tbtags files found!
DEBG: 12 .avro-tbtags files filled to list!
INFO: [0]: Restoring from ess_cloud.3438732572185.avro-tbtags ...
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732572185.avro-tbtags
INFO: [1]: Restoring from ess_cloud.3438732572316.avro-tbtags ...
INFO: [2]: Restoring from ess_cloud.3438732572343.avro-tbtags ...
INFO: [4]: Restoring from ess_cloud.3438732604114.avro-tbtags ...
INFO: [3]: Restoring from ess_cloud.3438732602635.avro-tbtags ...
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732572343.avro-tbtags
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732572316.avro-tbtags
INFO: [5]: Restoring from ess_cloud.3438732604138.avro-tbtags ...
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732602635.avro-tbtags
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732604114.avro-tbtags
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732604138.avro-tbtags
INFO: [6]: Restoring from ess_cloud.3438732604175.avro-tbtags ...
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732604175.avro-tbtags
INFO: [7]: Restoring from ess_cloud.3438732607035.avro-tbtags ...
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732607035.avro-tbtags
INFO: [8]: Restoring from ess_cloud.3438732607108.avro-tbtags ...
INFO: [9]: Restoring from ess_cloud.3438732609974.avro-tbtags ...
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732607108.avro-tbtags
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732609974.avro-tbtags
INFO: [10]: Restoring from ess_cloud.3438732612378.avro-tbtags ...
DEBG: dumpInOneAvroFile() LN7522, Namespace: ess_cloud
DEBG: dumpInOneAvroFile() LN7522, Namespace: ess_cloud
DEBG: dumpInOneAvroFile() LN7522, Namespace: ess_cloud
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732612378.avro-tbtags
INFO: [11]: Restoring from ess_cloud.3438732612455.avro-tbtags ...
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732612455.avro-tbtags
DEBG: dumpInOneAvroFile() LN7522, Namespace: ess_cloud
DEBG: dumpInOneAvroFile() LN7522, Namespace: ess_cloud
DEBG: dumpInOneAvroFile() LN7522, Namespace: ess_cloud
DEBG: dumpInOneAvroFile() LN7522, Namespace: ess_cloud

运行环境:

CentOS 7.9
Memory: 7.6G, CPU: Intel(R) Xeon(R) Silver 4214R CPU @ 2.40GHz * 4, current Disk Space: 184G
TDengine Version 3.2.3.0

Jun 26 '24 04:06 Whitroom