incubator-gluten icon indicating copy to clipboard operation
incubator-gluten copied to clipboard

[backend-Velox] Use tpch_convert_parquet_dwrf.sh to convert parquet but cause exception 'double free or corruption (!prev)'

Open KevinyhZou opened this issue 2 years ago • 1 comments

Describe the bug generate the parquet table file by use tpch-dbgen: https://github.com/databricks/tpch-dbgen

then convert the parquet file to dwrf format by use the tpch_convert_parquet_dwrf.sh script, and submit the spark job to yarn, the job fail and the error log as below gluten_bug_1

it seems the jni call make the exception

To Reproduce re-run tpch_convert_parquet_dwrf.sh script

Expected behavior convert parquet to dwrf without error

Additional context

KevinyhZou avatar Sep 01 '22 12:09 KevinyhZou

parquet is supported now. You may try parquet file directly.

FelixYBW avatar Oct 18 '22 18:10 FelixYBW

parquet is supported now. You may try parquet file directly.

OK

KevinyhZou avatar Oct 19 '22 07:10 KevinyhZou