Hongze Zhang
Hongze Zhang
I think maybe this sort of `column metadata` can be somehow centralized/reorganized in future, cause different consumer might care about different information about a specific column. E.g. compression, dictionary encoding,...
Would you rebase also? Thanks.
> In our case, we allocate more off-heap memory and less on-heap memory to Gluten compared to vanilla Spark. As a result, vanilla Spark can succeed with enough on-heap memory...
And CH CI failed https://opencicd.kyligence.com/job/gluten/job/gluten-ci/10302/, please also take a look. Thanks.
@ivoson Do you want to show some screenshots on Spark UI about the difference made by this PR? Thanks.
+1 to the idea. I remember there was vanilla Spark limitation as blocker against a similar effort. But let's see if it's doable at this time.
> @zhztheplayer if nobody is working on this, please let me know, I can add support for this. Thanks! Sure, feel free to take. Thank you for helping.
Thank you for this effort. It's critical for the stability of release 1.2. BTW, do you happen to know why `Velox backend Github Runner / run-tpc-test-ubuntu-oom` is failing? I will...
Run Gluten Clickhouse CI
Hi @wForget , thank you for keeping working on this. And I might forget that, have you picked https://github.com/apache/incubator-gluten/pull/7244? Which targeted to fix some hanging issues.