bug: tmp file can not be removed automatically after user load/import

[GreatSQL]

Search before asking

• [X] I had searched in the issues and found no similar issues.

Version

v1.2.371-a95ac62303(rust-1.77.0-nightly-2024-03-11T01:07:23.093484068Z)

What's Wrong?

tmp file can not be removed automatically after user load/import

How to Reproduce?

$ docker pull datafuselabs/databend $ docker run -itd --net=host datafuselabs/databend

init a tpc-h database and tables, and then load locale files, like this: bendsql -D tpch --query="INSERT INTO region values;" --format=csv --data=region.csv & bendsql -D tpch --query="INSERT INTO nation values;" --format=csv --data=nation.csv & bendsql -D tpch --query="INSERT INTO supplier values;" --format=csv --data=supplier.csv & bendsql -D tpch --query="INSERT INTO customer values;" --format=csv --data=customer.csv & bendsql -D tpch --query="INSERT INTO part values;" --format=csv --data=part.csv & bendsql -D tpch --query="INSERT INTO partsupp values;" --format=csv --data=partsupp.csv & bendsql -D tpch --query="INSERT INTO orders values;" --format=csv --data=orders.csv & bendsql -D tpch --query="INSERT INTO lineitem values;" --format=csv --data=lineitem.csv &

after all files had been loaded, there were some tmp file in /var/lib/databend in docker $ docker exec -it xx bash $ cd /var/lib/databend root@databend:/var/lib/databend$ du -sch query/* 178G query/29 52G query/stage 230G total

root@localhost:8000/tpch> list @~;

list @ ~

┌────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ name │ size │ md5 │ last_modified │ creator │ │ String │ UInt64 │ Nullable(String) │ String │ Nullable(String) │ ├─────────────────────────────────┼────────────┼──────────────────┼───────────────────────────────┼──────────────────┤ │ client/load/1710812774009926262 │ 64 │ NULL │ 2024-03-19 01:46:14.017 +0000 │ NULL │ │ client/load/1710813135483205998 │ 2199 │ NULL │ 2024-03-19 01:52:15.487 +0000 │ NULL │ │ client/load/1710814143019919868 │ 2446173319 │ NULL │ 2024-03-19 02:15:35.894 +0000 │ NULL │ │ client/load/1710814221250906168 │ 536839995 │ NULL │ 2024-03-19 02:11:53.716 +0000 │ NULL │ │ client/load/1710814211215622367 │ 536809803 │ NULL │ 2024-03-19 02:11:53.932 +0000 │ NULL │ │ client/load/1710814142997685956 │ 2429044244 │ NULL │ 2024-03-19 02:15:35.109 +0000 │ NULL │ │ client/load/1710814143008004170 │ 2447966144 │ NULL │ 2024-03-19 02:15:47.026 +0000 │ NULL │ │ client/load/1710814143008782376 │ 2446602335 │ NULL │ 2024-03-19 02:15:49.916 +0000 │ NULL │ │ client/load/1710814163120320536 │ 2446458285 │ NULL │ 2024-03-19 02:15:46.647 +0000 │ NULL │ │ client/load/1710814211222525112 │ 536808368 │ NULL │ 2024-03-19 02:11:58.254 +0000 │ NULL │ │ client/load/1710814221247956086 │ 536836457 │ NULL │ 2024-03-19 02:12:02.437 +0000 │ NULL │ │ client/load/1710814211317297839 │ 536835128 │ NULL │ 2024-03-19 02:12:02.731 +0000 │ NULL │ │ client/load/1710814221241450890 │ 536901229 │ NULL │ 2024-03-19 02:12:39.411 +0000 │ NULL │ │ client/load/1710814201205750513 │ 536923067 │ NULL │ 2024-03-19 02:12:21.237 +0000 │ NULL │ │ client/load/1710814221253312105 │ 536864660 │ NULL │ 2024-03-19 02:12:03.793 +0000 │ NULL │ │ client/load/1710814201188227093 │ 536874153 │ NULL │ 2024-03-19 02:12:20.863 +0000 │ NULL │ │ client/load/1710814201188760171 │ 536938129 │ NULL │ 2024-03-19 02:12:23.820 +0000 │ NULL │ │ client/load/1710814211218967529 │ 536810162 │ NULL │ 2024-03-19 02:12:07.724 +0000 │ NULL │ │ client/load/1710814231273670802 │ 536895531 │ NULL │ 2024-03-19 02:12:26.272 +0000 │ NULL │ │ client/load/1710814211225863115 │ 536871997 │ NULL │ 2024-03-19 02:12:07.971 +0000 │ NULL │ │ client/load/1710814211307879050 │ 536864111 │ NULL │ 2024-03-19 02:12:08.518 +0000 │ NULL │ │ client/load/1710814221238471854 │ 536885544 │ NULL │ 2024-03-19 02:12:34.677 +0000 │ NULL │ │ client/load/1710814231304333058 │ 536819687 │ NULL │ 2024-03-19 02:12:10.355 +0000 │ NULL │ │ client/load/1710814143001383191 │ 2428331709 │ NULL │ 2024-03-19 02:15:16.922 +0000 │ NULL │ │ client/load/1710814142999026371 │ 2426794146 │ NULL │ 2024-03-19 02:15:18.120 +0000 │ NULL │ │ client/load/1710814143005684920 │ 2417035971 │ NULL │ 2024-03-19 02:15:34.184 +0000 │ NULL │ │ client/load/1710814143006045398 │ 2436766541 │ NULL │ 2024-03-19 02:15:35.352 +0000 │ NULL │ │ client/load/1710814163129753599 │ 2013143040 │ NULL │ 2024-03-19 02:17:28.214 +0000 │ NULL │ │ client/load/1710814153015166473 │ 2119262208 │ NULL │ 2024-03-19 02:17:28.429 +0000 │ NULL │ │ client/load/1710814221255374289 │ 536808753 │ NULL │ 2024-03-19 02:11:51.728 +0000 │ NULL │ │ client/load/1710814143005583902 │ 2428896542 │ NULL │ 2024-03-19 02:15:17.160 +0000 │ NULL │ │ client/load/1710814143013780006 │ 2446881757 │ NULL │ 2024-03-19 02:15:25.120 +0000 │ NULL │ │ client/load/1710814153021763623 │ 2448125647 │ NULL │ 2024-03-19 02:17:24.632 +0000 │ NULL │ │ client/load/1710814153037342291 │ 2237661184 │ NULL │ 2024-03-19 02:17:21.598 +0000 │ NULL │ │ client/load/1710814153035568809 │ 2446482480 │ NULL │ 2024-03-19 02:17:24.055 +0000 │ NULL │ │ client/load/1710814153026589118 │ 2132803584 │ NULL │ 2024-03-19 02:17:21.952 +0000 │ NULL │ │ client/load/1710814153022466971 │ 1849278464 │ NULL │ 2024-03-19 02:17:28.303 +0000 │ NULL │ │ client/load/1710814153032639267 │ 2261139456 │ NULL │ 2024-03-19 02:17:28.192 +0000 │ NULL │ │ client/load/1710814153039715792 │ 1581711360 │ NULL │ 2024-03-19 02:17:28.165 +0000 │ NULL │ └────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

39 rows read in 0.042 sec. Processed 39 rows, 3.85 KiB (935.22 rows/s, 92.31 KiB/s)

root@localhost:8000/tpch> remove @~;

remove @ ~

0 row read in 0.084 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

root@localhost:8000/tpch> list @~;

list @ ~

0 row read in 0.035 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

[everpcpc]

Please check if there were errors occurred during insert. Stage file would not be deleted if the insert fails.