tidb
tidb copied to clipboard
pingcap
Lightning encounters OOM when importing Parquet files
Bug Report
Please answer these questions before submitting your issue. Thanks!
1. Minimal reproduce step (Required)
Lightning imports 100 parquet files, each has a size of 278.1MB and consists of 1,000,000 rows.
2. What did you expect to see? (Required)
No error.
3. What did you see instead (Required)
fatal error: runtime: out of memory
4. What is your TiDB version? (Required)
v7.5.0
@lyzx2001: The label(s) sig/affects-7.5 cannot be applied, because the repository doesn't have them.
In response to this:
/sig affects-7.5
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.
![ti-chi-bot[bot] avatar](https://avatars.githubusercontent.com/in/214286?v=4 "Published by a pub.dev verified publisher"){kind=small}
@lyzx2001: The label(s) sig/affects-7.5 cannot be applied, because the repository doesn't have them.
In response to this:
/sig affects-7.5
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.
@lyzx2001: The label(s) affects-7.4 cannot be applied. These labels are supported: fuzz/sqlancer, challenge-program, compatibility-breaker, first-time-contributor, contribution, good first issue, correctness, duplicate, proposal, security, ok-to-test, needs-ok-to-test, needs-more-info, needs-cherry-pick-release-5.4, needs-cherry-pick-release-6.1, needs-cherry-pick-release-6.5, needs-cherry-pick-release-7.1, needs-cherry-pick-release-7.5, affects-5.4, affects-6.1, affects-6.5, affects-7.1, affects-7.5, may-affects-5.4, may-affects-6.1, may-affects-6.5, may-affects-7.1, may-affects-7.5.
In response to this:
/label affects-7.4
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.
@lyzx2001: The label(s) affects-5.3 cannot be applied. These labels are supported: fuzz/sqlancer, challenge-program, compatibility-breaker, first-time-contributor, contribution, good first issue, correctness, duplicate, proposal, security, ok-to-test, needs-ok-to-test, needs-more-info, needs-cherry-pick-release-5.4, needs-cherry-pick-release-6.1, needs-cherry-pick-release-6.5, needs-cherry-pick-release-7.1, needs-cherry-pick-release-7.5, affects-5.4, affects-6.1, affects-6.5, affects-7.1, affects-7.5, may-affects-5.4, may-affects-6.1, may-affects-6.5, may-affects-7.1, may-affects-7.5.
In response to this:
/label affects-5.3
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.