[Bug] 导出tsfile总是出现内存不足的问题

[Wang-0122]

Search before asking

• [X] I searched in the issues and found nothing similar.

Version

Cannot dump result because: 301: There is not enough memory to execute current fragment instance, current remaining free memory is 44350172B, estimated memory usage for current fragment instance is 848419096B

内存是足够的, 是不是要修改什么配置的, 2c8g的机器

Describe the bug and provide the minimal reproduce step

1

What did you expect to see?

1

What did you see instead?

1

Anything else?

No response

Are you willing to submit a PR?

• [ ] I'm willing to submit a PR!

[Wang-0122]

版本: 1.3.3, 1个database, 大部分类型都有 集群环境为 3C3D

[CritasWang]

可以修改datanode-env.sh 的 memory-size 如果 是 windows 则是 datanode-env.bat

[Wang-0122]

可以修改datanode-env.sh 的 memory-size 如果 是 windows 则是 datanode-env.bat

8G内存的机器, MEMORY_SIZE=6G 已经改到6G了, 还是不行 参考了这个链接做了些修改, 也无济于事 https://iotdb.apache.org/zh/UserGuide/latest/FAQ/Frequently-asked-questions.html#%E9%A2%84%E4%BC%B0%E5%86%85%E5%AD%98%E4%B8%8D%E8%B6%B3%E6%8A%A5%E9%94%99%E5%A6%82%E4%BD%95%E5%A4%84%E7%90%86

[root@ip-10-0-137-177 apache-iotdb-1.3.3-all-bin]# tools/export-tsfile.sh -h 127.0.0.1 -p 6667 -u root -pw root -t ./backup -q "select * from root.vGco.** where time >= 1736734200000 and time <= 1736734260000"

Starting IoTDB Client Export Script

Cannot dump result because: 301: There is not enough memory to execute current fragment instance, current remaining free memory is 308727591B, estimated memory usage for current fragment instance is 888118024B

只导出1分钟的数据也不行, 需要888118024B = 800M 内存

[TheSeasonForHer]

1、maybe you do not restart service after you edit the varible 参考1 2、you can consider that use ‘‘align by device“ adding in the sql or slimit 参考2

[CritasWang]

可以修改datanode-env.sh 的 memory-size 如果 是 windows 则是 datanode-env.bat

8G内存的机器, MEMORY_SIZE=6G 已经改到6G了, 还是不行 参考了这个链接做了些修改, 也无济于事 https://iotdb.apache.org/zh/UserGuide/latest/FAQ/Frequently-asked-questions.html#%E9%A2%84%E4%BC%B0%E5%86%85%E5%AD%98%E4%B8%8D%E8%B6%B3%E6%8A%A5%E9%94%99%E5%A6%82%E4%BD%95%E5%A4%84%E7%90%86

[root@ip-10-0-137-177 apache-iotdb-1.3.3-all-bin]# tools/export-tsfile.sh -h 127.0.0.1 -p 6667 -u root -pw root -t ./backup -q "select * from root.vGco.** where time >= 1736734200000 and time <= 1736734260000"

Starting IoTDB Client Export Script

Cannot dump result because: 301: There is not enough memory to execute current fragment instance, current remaining free memory is 308727591B, estimated memory usage for current fragment instance is 888118024B

只导出1分钟的数据也不行, 需要888118024B = 800M 内存

这个操作中会将root.vGco.** 内所有的测点按时间对齐组成一个超级大宽表，因此预估的资源会比较多。 建议使用 export-tsfile.sh 时 select 语句中仅包含单个 device，不同的 device 分开来导出