br icon indicating copy to clipboard operation
br copied to clipboard

Lightning supports parallel import of large-scale data (100T+)

Open IANTHEREAL opened this issue 4 years ago • 0 comments

Feature Request

Describe your feature request related problem:

Import 100T+ data quickly and easily! Currently Lightning is a single-process tool, and the import speed of a single process has an upper limit. It does not support the import of large-scale data (100T+) or is not easy to use

Describe the feature you'd like:

Lightning can support distributed parallel import

Describe alternatives you've considered:

Integrated distributed processing system, such as spark, etc.

Teachability, Documentation, Adoption, Migration Strategy:

IANTHEREAL avatar Jun 03 '21 08:06 IANTHEREAL