Samuel Colvin

The v1 shim will be removed in v3, not before.

Thanks @frankie567 we'll look into this, I think @alexmojaki has something similar working until we can support that syntax in datafusion.

I've created https://github.com/apache/datafusion/issues/12533, and commented on https://github.com/apache/datafusion/issues/11732. Once I get some guidance on next steps, I'll try to work on it.

Well that's embarrassing 🤦, I forgot `--release`. In release mode: * unsorted string: 82ms * unsorted FWB: 82ms * sorted string: 11ms * sorted FWB: 82ms Kind of even more...

Well it keeps getting weirder. In release mode: * unsorted string: 82ms * unsorted FWB: 82ms * unsorted UInt64: 51ms * sorted string: 11ms * sorted FWB: 82ms * sorted...

Okay last comment here (for now), I'll stop talking to myself. It seems that `Decimal128` is the best option for our case (we can rewrite it to look like hex...

What's weird is the behaviour with a decimal 128 is better than a uint64 when sorted. Is that that a fundamental side affect of the type, or some missing logic/optimisation?

I think the problem is that [here](https://github.com/apache/arrow-rs/blob/f42d2420525a05a9b55461d83b359779ca5cc2a3/arrow-select/src/filter.rs#L320-L383) `arrow-rs` has special cases for primative types and "byte types" e.g. strings, but no special case for `FixedSizeBinary`. Not sure how much of...

Closed

Principle here looks good, but there's lots of conflicts and I want to change the logic a little, so I'll create a new PR to fix this.