Hongze Zhang

comment to keep the PR open as it could be a valuable topic

Thank you! BTW there were a couple of related efforts in our code base (not all of them): https://github.com/apache/incubator-gluten/pull/432 https://github.com/apache/incubator-gluten/pull/5278 Should we review them then remove the unnecessary / unmaintained...

Which library Velox's lib loader tries to unload? I remember that option (`boolean requireUnload`) was only used by CH backend.

> > Which library Velox's lib loader tries to unload? I remember that option (`boolean requireUnload`) was only used by CH backend. > > I'm not sure about which libs...

@yangzhg I could not reproduce the unloading error since the static storage `REQUIRE_UNLOAD_LIBRARY_PATHS` looks to be always empty in Velox backend thus the shutdown hook should ideally do nothing. ...

> Add configuration spark.memory.storage.preferSpillNative to determine preference of spilling RDD cache or FileCache(Native) when storage memory should be shrinked. For example, when queries are mostly executed on same data sources,...

Do we need to have a test for Scala 2.13 (with Spark 3.5 probably) ? https://github.com/apache/incubator-gluten/blob/main/.github/workflows/velox_docker.yml

Run Gluten Clickhouse CI

> GlutenConfig prevents the dynamically modified configurations in interactive scenarios. @beliefer Would you like to post a code example of the interactive case you mentioned? Or probably to add a...

@beliefer Aside of the constructor changes for `GlutenConfig`, I think 2 parts of code cleanups can be split and merged first. Would you want to split out the following parts...