Andrew Lamb

I think the first thing to do would be to try and write some tests that show the error happening Perhaps we could use the existing statistics: `predicate_evaluation_errors` https://github.com/apache/datafusion/blob/6d5e00ad3f8e53f7252cb1d3c72a6c7f28c1aed6/datafusion/datasource-parquet/src/metrics.rs#L31-L30 Here...

> What about in the situation described above? What happens now is basically that both columns get cast to Int32 to match their table level types. But that's at least...

> variability comes from reading the data, parsing parquet, etc. and once it's in memory as arrow converting from Int32 to Int64 is tri This makes a lot of sense...

I plan to review this tomorrow

FYI @phillipleblanc

thanks @MohamedAbdeen21 -- I am very behind on reviews. I put this one on my list again

cc @backkem and @devinjdangelo

Here is a PR to improve the documentation https://github.com/apache/datafusion/pull/11395 (based on this one)

Thanks again @MohamedAbdeen21

Likely cause: - https://github.com/apache/datafusion/pull/16234 Revert PR: - https://github.com/apache/datafusion/pull/16307