Struct/Map support roadmap

[Seaven]

Backgroud

At present, starrocks already supports static Struct/Map types in the external table, and can query external tables (hive/parquet/....) through related expressions. Some user will load the external data into starrocks, but starrocks has not supported the storage and import process of Struct/Map in the OLAP table.

Target

1. Internal tables support DDL operations on static Struct/Map type columns
2. Support importing Struct/Map types from external data sources to starrocks
3. Support querying Struct/Map and related expressions
4. Optimize Struct/Map query performance as much as appearance

Currently supported

1. Struct/Map storage of internal tables (the code is ready, but needs to be test & review)
2. Memory column structure of Struct/Map
3. Some simple expressions

Phase 1.

1. Internal tables support DDL operations on static Struct/Map type columns

• [x] Table schema create
• [x] Stuct/Map type syntax

2. Support load Struct/Map types

• [x] Data source format (Parquet, ORC)
• [x] Import ways (Insert Into, Broker Load)
• [x] Test & review Struct/Map storage of internal tables

3. Optimize Struct/Map query performance

• [x] Struct/Map column access performance optimization
• [x] Structure internal column prune design & implement

Phase 2.

1. Internal tables support DDL operations on static Struct/Map type columns

• [ ] Table schema change

2. Support load Struct/Map types

• [ ] Data source format (CSV)
• [ ] Import ways (Stream Load, Rotinue Load)

3. Support querying Struct/Map and related expressions

• [x] Struct/Map data type conversion

4. Optimize Struct/Map query performance

• [ ] Expression performance optimization

other