Sem

@danielbeach Are you still working on it? May I help here somehow?

@gengliangwang May you assign this one to me please? I would like to try to do it.

> @SemyonSinchenko +1, this is a great idea. We can avoid duplicated code on doc website and example notebooks. Can I assign this one to you? Yes, I can try!

> Hmm, it looks like Databricks Spark changes the constructor of FileSourceScanExec? But we cannot look at Databricks Spark source code. I can raise this question to Databricks support. I...

I tried and error was changed :) ``` com.databricks.backend.common.rpc.SparkDriverExceptions$SQLExecutionException: java.lang.NoSuchMethodError: org.apache.spark.sql.execution.DataSourceScanExec.$init$(Lorg/apache/spark/sql/execution/DataSourceScanExec;)V at org.apache.spark.sql.comet.CometScanExec.(CometScanExec.scala:63) at org.apache.spark.sql.comet.CometScanExec$.apply(CometScanExec.scala:476) at org.apache.comet.CometSparkSessionExtensions$CometScanRule$$anonfun$apply$5.applyOrElse(CometSparkSessionExtensions.scala:127) at org.apache.comet.CometSparkSessionExtensions$CometScanRule$$anonfun$apply$5.applyOrElse(CometSparkSessionExtensions.scala:77) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:465) at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(origin.scala:69) ``` I have zero ideas whats wrong...

Databricks support told me that DBR 13.3 should be fully compatible with all the public API of OSS Spark, but I guess that they backported some patches from 3.5. I...

DBR 13.3 [uses](https://docs.databricks.com/en/release-notes/runtime/13.3lts.html#system-environment) 2.12.15; Comet uses 2.12.17... May the problem be here? I can try to build comet with 2.12.15, just change in one line in `pom.xml`

Can I make a PR?

What if I create something like spark build info? It looks like only [branch, build, commit and remote information](https://github.com/apache/arrow-datafusion-comet/blob/main/common/pom.xml#L98) are included. In this case such a bash script may be...

This short bash script can generate everything like in `common/target/classes/comet-git-info.properties`: ```bash #!/usr/bin/bash echo_build_properties() { echo "git.branch=$(git rev-parse --abbrev-ref HEAD)" echo "git.build.host=$(uname -n)" echo "git.build.time=$(date -u +%Y-%m-%dT%H:%M:%S%z)" echo "git.build.user.email=$(git config user.email)"...