QP Hou

off topic, but in case you missed it, there is also a fairly uptodate arrow2 branch for datafusion that's being worked on.

I think this is a great idea. Promote mutable array into the core, then build immutable apis as thin wrappers unlocks a lot of optimization opportunities :+1:

oh, nice, that's less work for us :D cc @yjshen

I agree, it would be better to track memory usage in downstream compute engines by calculating physical memory consumption including all fields in the array structs.

Do we need to consider or measure the overhead for accessing stack v.s. heap data? As for expensive clone, I wonder if it would work better if we let users...

Good call, I agree adding Arc on individual arm would be more efficient than `Arc`.

https://github.com/gobwas/glob also supports `**`, so that's a good candidate to benchmark against.

Yes, I do.

Perhaps my comment should be tracked as a separate issue, but I think what would be really useful is to generate an interactive zoomable svg diagram like what typical flamegraph...

Thank you @liukun4515 ! In case you missed it, the setup I had in https://github.com/apache/arrow-datafusion/pull/688 already works and was only paused pending optimizations to reduce cache size. I imagine you...