Geofrey Ernest

With the new api for building `arrow.Record` We already keep tab on dynamic columns per `.Append` so its pretty much there already. Considering `.Append` works with batches, ` proto.Merge` won't...

> I don't think I understand why this is the case. The way I envisioned it is that the concrete dynamic columns will be stored as part of the table...

Building records directly is simple. However this requires sorting `arrow.Record` before inserting to the table. Sorting is not implemented in arrow package yet. I'm contemplating using normal `sort.Interface` to build...

I am thinking about this, I was wondering if this is the same as `arrowutils.MergeRecords(arrow_parts...) |> arrowutils.SortRecord |> parts.NewArrowPart` ?

@asubiotto can you expand a bit about memory expectation between arrow/parquet compaction ? I was always under the impression `parquet+compression` gives better memory saving than `arrow`.

@cznic @devfacet I would like to look into this. Is this still an I issue. If thee is already an ongoing effort, I will be more than happy to help....

gentle ping @cznic can yuo please tell me what is the expected result from this ? ```sql drop table if exists category; drop table if exists condition; drop table if...

@cznic no problem . I think I found out a way to solve this. I fixed the fields now i get ``` "product_key", "category_name", "product_condkey", "condition.key", "condition.name" ``` I will...

another gentle ping @cznic @devfacet On my dev branch ``` $ ql -fld < join.ql "product_key", "category_name", "product_condkey", "condition.key", "condition.name" 4, , , , 3, , 2, 2, "baz" 2,...

@devfacet thanks for the recommendation. At the moment I would like to improve ql hopefully it will be production ready one day.