asbisen

I am noticing something similar. Using `polars.read_ipc()` works but `polars.scan_ipc()` with a list of arrow files as the argument fails on parsing Julia DateTime format.

Nope I only see this issue on `jupyter-notebook` after upgrading my install; `PlotlyJS.jl` is not longer working with `jupyter-lab` with recent updates. I am not sure what broke the install,...

quite possibly it's coming from here https://github.com/JuliaGizmos/JSExpr.jl/blob/master/src/JSExpr.jl#L67. casually changing scope.id to scopeid(scope) does not fix the problem though.

Created a pull request https://github.com/JuliaStats/MLBase.jl/pull/46 that closes this issue.

Arrow support would allow for much better overall performance. DuckDB already provides the ability to create [relations from Arrow Table](https://github.com/duckdb/duckdb/blob/0d3fee81ba83d69adf362d92482101a82a4a5de1/tools/pythonpkg/duckdb_python.cpp#L122)

I don't think I have the technical chops for that (calling c interfaces from Julia), but I can certainly give it a shot and see how it goes.

@hannes I am assuming that if we can pass results in Arrow directly to [Arrow.Table()](https://arrow.juliadata.org/dev/manual/#Arrow.Table) then we can avoid materializing the results in DataFrames.jl and allocating more memory. Then the...

@kimmolinna @hannes I guess I used a wrong analogy, I use `Query.jl` extensively mostly for feature engineering where I can use custom functions written in Julia. Which becomes cumbersome with...

I have unintentionally managed to divert the attention from exposing `Arrow` pointer to `Tables.jl` interface. I see a big value in exposing the results from DuckDB as pointer to Arrow...

`Arrow.jl` exposes `Tables.jl` interface too. So once `Arrow.jl` get's the result from DuckDB those tables can be accessed by any package/program. For example I can use [Dagger.jl](https://github.com/JuliaParallel/Dagger.jl) to process the...