Ritchie Vink
Ritchie Vink
I am thinking about a `Expr.meta -> MetaExpr` namespace. This can implement the magic methods on a meta level. E.g. comparing expressions by expression tree. We can also add methods...
There is an example using the arrow c data interface here: https://github.com/pola-rs/polars/tree/master/examples/python_rust_compiled_function But now that I think of it. We could actually provide a library that does the conversion for...
Thanks @matteosantama. I am a few days off, but I will review this as soon as possible.
@matteosantama I did a first review and the code seems good to me. Can you finish you todo's and pnig me a again for a final review?
@matteosantama this can be closed now right?
I strongly feel that this should not be handled by polars. If one wants to export to datatypes not supported by polars they can convert the polars `DataFrame` to arrow...
We only support local file systems. Remote storage blob is not in scope for the project (yet).
> Doesn't Polars support reading from Read + Seek? For csv and parquet we use memmap. > There is a semantic relationship between Read + Seek and [byte-range headers](https://docs.microsoft.com/en-us/rest/api/storageservices/specifying-the-range-header-for-blob-service-operations#range-header-formats) supported...
Downloading the file once and cache it. We are unable to do anything smart with blob storage. In the python bindings we use `fsspec` to get cloud files. That gives...
> But I may be missing something here - isn't the parquet reader code using std::fs::File and passing it to arrow2? Something like ``` enum Input { PathOrGlob(String), File(Box), }...