iotdb icon indicating copy to clipboard operation
iotdb copied to clipboard

import-csv.bat导入数据时提示路径全数字问题

Open mnote opened this issue 3 years ago • 2 comments

windows下执行以下命令导入路径(root.sg.T01601)的数据时,有三个小问题:

set JAVA_HOME=D:\jdk8_x64
D:/iotdb/tools/import-csv.bat -h 127.0.0.1 -p 6667 -u root -pw root -aligned false -batch 100000 -f D:\20211207\20211207_T01601.csv -fd D:\20211207\failed\ > log.txt

(1)import-csv.bat会执行select * from root.sg limit 1,导致服务端报错:

2022-08-01 21:34:00,922 [pool-16-IoTDB-RPC-Client-13] ERROR o.a.i.d.u.ErrorHandlingUtils:83 - Status code: 411, Query Statement: "select * from root.sg limit 1". executeQueryStatement failed
org.apache.iotdb.db.exception.query.PathNumOverLimitException: Too many paths in one query! Currently allowed max deduplicated path number is 1000. Please use slimit or adjust max_deduplicated_path_num in iotdb-engine.properties.

(2)import-csv.bat提示全数字路径(purely digital path),实际的路径(root.sg.T01601)不是全数字 import-csv.bat执行的结果log.txt文件内容:

Starting IoTDB Client Import Script

21:34:00.893 [main] DEBUG org.apache.iotdb.session.Session - EndPoint(ip:127.0.0.1, port:6667) execute sql select * from root.sg limit 1 Meet error when query the type of timeseries because the IoTDB v0.13 don't support that the path contains any purely digital path. 21:34:01.626 [main] DEBUG org.apache.iotdb.rpc.AutoResizingBuffer - org.apache.iotdb.rpc.AutoResizingBuffer@50d0686 expand from 1024 to 1536, request: 1026 21:34:01.627 [main] DEBUG org.apache.iotdb.rpc.AutoResizingBuffer - org.apache.iotdb.rpc.AutoResizingBuffer@50d0686 expand from 1536 to 2304, request: 1543 ...... 21:34:01.740 [main] DEBUG org.apache.iotdb.rpc.AutoResizingBuffer - org.apache.iotdb.rpc.AutoResizingBuffer@50d0686 expand from 3405019 to 5107528, request: 3405023 Import completely!

(3)import-csv.bat为什么需要执行select * from root.sg limit 1,感觉这里不需要,路径的类型已经在csv文件里第一行有了

20211207_T01601.csv文件内容:

Time,root.sg.T01601(FLOAT) 2021-12-07T00:00:00.000+0800,1.6395046432176699 2021-12-07T00:00:01.000+0800,52.37059846464655 2021-12-07T00:00:02.000+0800,245.19064110874533 ......

Desktop (please complete the following information):

  • OS: Windows 2012,iotdb 0.13.1

mnote avatar Aug 01 '22 13:08 mnote

Thanks for your feedback! We'll check the problems you mentioned and fix them in next version.

HTHou avatar Aug 03 '22 01:08 HTHou

问题1 可以通过提示修改server的配置参数解决 问题3 是一个优化点,可以在提供了数据类型的时候不执行查询,我们会做这样的优化 问题2 我使用样例csv文件没有复现,方便可以提供一下报错的csv文件

Q1. You can try to modify the configuration that the log mentioned. Q2. It didn't reproduce when I use the sample csv file. Can you please provide the csv that encounterd the error? Q3 is good point. We will do the optimization.

HTHou avatar Aug 03 '22 06:08 HTHou