Jay Chia

@savingoyal 1. **Benchmarking**: do we have existing benchmark FlowSpecs that I can run on the sandbox if I can be granted access to one? I can benchmark this against running...

> @jaychia: Is the Datastore Stable? I have built a [Kubernetes Plugin](https://github.com/valayDave/metaflow/tree/kube_cpu_stable) on my version of the Metaflow's Fork. I have not used ur plugin yet, But you can try...

> This is awesome. Just wanted to check in on the status and if this is still something you're looking to integrate in the near future. @jaychia Yup! You can...

Yes indeed - Daft is probably most similar to the Polars lazy API. We do not yet have a SQL frontend. Would https://github.com/ibis-project/ibis/blob/main/ibis/backends/polars/tests/conftest.py be a good place to start to...

I've been doing more digging into parquet-format and versioning of the spec has definitely been one of the more confusing pieces. I'm glad that there is an effort to define...

> Nice! This looks like a great starting point - let me do the work we discussed on my end and (1) cleanup unused functions and (2) move pandas/python specific...

Quick question @arnavgarg1 - do you know how `.map_partitions` and `.map_batches` ([on the DataFrameEngines](https://github.com/ludwig-ai/ludwig/blob/master/ludwig/data/dataframe/base.py#L47-L53)) are different? Based on the function signature/naming it seems like the intention is for `.map_partitions` to...

A quick update here: The team is actively looking at building up a SQL frontend to Daft. We have basic support up already, with a more extensive roadmap detailed here:...

No specific dialect at the moment, we're still building out SQL support in Daft and can provide more updates as we go along. IIUC then if we are compatible with...

Note also that the object_store crate might already support this functionality but under a different key name: https://docs.rs/object_store/latest/object_store/aws/enum.AmazonS3ConfigKey.html#variant.SkipSignature