Andy Grove
Andy Grove
Thanks @davidlghellin. Related issue: https://github.com/apache/datafusion-comet/issues/2084
I have a smaller repro: ```scala test("sort on timestamp after changing session timezone") { // create data in specific timezone withSQLConf(SQLConf.SESSION_LOCAL_TIMEZONE.key -> "Europe/London") { createTimestampTestData.createOrReplaceTempView("tbl") } // read data in...
> just FYI the workaround is > > ``` > CometConf.COMET_EXEC_SHUFFLE_ENABLED.key -> "false" > ``` > > This happens because in RangeRepartition on RowConverter, I'm digging into this real quick...
short term fix may be to add some casting to UTC
I think I have convinced myself that this is only an issue when using `spark.comet.sparkToColumnar.enabled` or other uses of `SparkToColumnar`, so I don't think it is urgent to fix for...
> I think I have convinced myself that this is only an issue when using `spark.comet.sparkToColumnar.enabled` or other uses of `SparkToColumnar`, so I don't think it is urgent to fix...
I propose adding these methods to `CometOperatorSerde`: ```scala def getExecConfigName(expr: T): String = expr.getClass.getSimpleName.replace("Exec", "") def getExecLegacyConfigName(expr: T): Option[String] = None ``` The current `enabledConfig` method can be removed: ```...
`GenerateDocs` will also need updating
> In the past I think we've encountered differences in Java and Rust's regex engines wrt graphemes. Could we get some larger UTF-8 characters in the tests? We probably need...
Also `serde.rs` `to_arrow_datatype` has hard-coded `UTC`.