Adam Binford

@zsxwing let me know what you think of this. I was hoping to potentially support updating the null checks to properly check the way Spark handles things, but that was...

Yeah this was just kinda brainstorming an approach. At first I tried to just fully prune any expressions to only the field it cared about, but then just hit weird...

I guess another strategy could be gather all attributes from the actual filters and replace them in the prepared filters by expr ID?

Also, I don't think `getScanReport` can catch this (after trying and seeing it not work). There's still only one FileSourceScanExec, it just does double duty in the `matchingFiles` method

Ah shoot not as easy as I thought, the ordinals for the struct fields will be off after that. Need to think through that alternative a little more

I think you would essentially have to "reresolve" the prepared filters so to speak, so the options I can think of right now are basically: - This approach, convert prepared...

The "unresolve both sides" seems less likely to have any weird resolution issues, but obviously just takes one example to prove it doesn't work. Haven't thought of one yet though

I believe it's the `NestedColumnAliasing` extractor in the `ColumnPruning` rule. So elaborating on the first two, these seem like the possibilities: 1. Expand on the current method a bit. Take...

Tried the first method, let me know what you think

No problem! I'd rather have a quicker release for Spark 3.3 😊