kudu icon indicating copy to clipboard operation
kudu copied to clipboard

[KUDU-3054][SPARK]Init kudu.write_duration accumulator lazily

Open liupc opened this issue 5 years ago • 2 comments

Currently, we encountered a issue in kudu-spark that will causing the following spark sql query failure:


Job aborted due to stage failure: Total size of serialized results of 942 tasks (2.0 GB) is bigger than spark.driver.maxResultSize (2.0 GB)

After carefully debug, we find out that it's the kudu.write_duration accumulators causing single spark task larger than 2M, thus all tasks size of the stage will bigger than the limit.

However, this stage is just reading kudu table and do shuffle exchange, no writing any kudu tables.

So I propose to init this accumulator lazily in KuduContext to avoid such issues.

liupc avatar Feb 14 '20 06:02 liupc

Hi @liupc.

Thank you for the pull request. The Kudu project doesn't merge new code through PRs; we use the gerrit tool to do code review and submission. Could you please follow the directions here to adapt this PR into a gerrit change?

adembo avatar Feb 14 '20 18:02 adembo

@adembo Thanks, I will try it later

liupc avatar Feb 17 '20 02:02 liupc