Marko Grujic

Hey @backkem, that's a good question. We abide by the `TableProvider` API set out by DataFusion which doesn't take into account the `ORDER BY` clause: https://github.com/splitgraph/seafowl/blob/fdd7c4996f9f06385e1d3c398f19c51929ea6c41/datafusion_remote_tables/src/provider.rs#L120-L126 Sorting itself is handled...

Not at the moment or in the near-term future, but it could be something we'd want to add later on (sorry for a bit of a late response).

Not really; note that the delta-rs migration effectively only changed the storage layer behind DataFusion's `TableProvider::scan`/`ExecutionPlan::execute`, and doesn't offer some primitives/hooks for query `changed / new / deleted results` type...

> so then you can get the the middle and correlate what data changes against what your uses are currently looking out that they did a query for a minute...

Aha no worries, I think we're (getting) on the same page. > I wonder if there is a way to track the version of a query and the data as...

Good question. My idea was to first make it available independently of Seafowl, which was done by separating it into a [standalone crate](https://github.com/splitgraph/seafowl/tree/main/datafusion_remote_tables). You should be able to add it...

@ccollie sure thing; I opened a PR to bump the dependencies over at connector-x: https://github.com/sfu-db/connector-x/pull/518 Hopefully this can result in a new `connector-x` release.

> I wonder if running the ARM binary inside [qemu](https://wiki.qemu.org/Documentation/Platforms/ARM) could help us do a basic 'did it run' test That's an interesting suggestion, thanks. Nonetheless, it seems simpler to...

I'm not sure why the [`parse decimal overflow`](https://github.com/delta-io/delta-rs/actions/runs/9004694154/job/24740575061?pr=2491#step:9:680) appears in the Python tests. It's indicative that I see the decimal column min/max ends up as `Value(Number(2560.0))`/`Value(Number(3584.0))` even though the pyarrrow...

Hi @roeap, @ion-elgreco, I think this is ready for review. There's both a feat added (convert stats) as well as a bugfix (decimal stats parsing from byte arrays) now. Can...