Renjie Liu
Renjie Liu
I think the next tasks will be: - Add reader that reads parquet into arrow. - Complete the converter to convert arrow schema to parquet schema. - Add writer to...
@andygrove Yes, I'll take that as a reference. Also I'll also reference the cpp implementation of arrow adapter of parquet.
@andygrove @sunchao Are you guys working on this? I'm working on an implementation which takes the cpp version as a reference.
I'm working on the arrow part. Yes only part of cpp version can be implemented. I'll work out an early version. Chao Sun 于 2018年10月10日周三 下午5:27写道： > @liurenjie1024 are you...
I'm working on an arrow reader implementation and have finished the first step, converting parquet schema to arrow schema in this [PR](https://github.com/sunchao/parquet-rs/pull/185), please help to review this.
I also have some thoughts about a unified execution engine, welcome to take a look and comment: https://www.notion.so/liurenjie1024/A-Cloud-Native-Universal-Execution-Engine-7903dd9eeea143c48049631a2d1cb845 cc @andygrove @mingmwang
Yes,deciding the execution mode(batched or pipelined) at runtime is an interesting topic, and I believe the first step is to make ballista flexible enough to support complex job graph.
这个项目的意义是什么呢？为什么不直接使用apache calcite呢？
> It is doable, but one question is: should we just move to sqllogictest now? Some test cases need to reference `data` directory to pre insert data. If we migrate...
Yes, we have discussed this in early days. DML in our system are used mostly for demo/test case, so concurrency control is not important for us.