Fix dynamic schema scan

[thorfour]

What if we had schema scans actually build a schema instead of returning a single column of name?

This allows us to do things like schema scans with dynamic projections since our query behavior is based on column names. Otherwise to support something like this we'd need to have each expr know if it's apart of a schema scan to behave differently.

[metalmatze]

Very much agree. The schema and schema iterator are currently in a weird state. We still have the SchemaIterator but use the ArrowSchema a lot more. Maybe we should more broadly speaking look at these again and combine the interfaces?

[brancz]

I was pretty sparse on the initial implementation but the idea was that the rows in the results are the columns of the schema, so an additional column of this result could for example be the type of the column. The whole point was that we could do things such as what you can do in postgres with select * from information_schema.columns where table_name = 'stud1';. The problem is though that we don't actually have a global schema so we will always need a layer that aggregates the potentially different sets of schemas, which is exactly what the distinct query plan does.

The idea was that the interface is that everything produces an arrow.Record, and schema scans are like accessing a different (meta-)table, therefore the expressions should not need to know anything special in regards to whether it's a table-scan or a schema-scan.

[thorfour]

I was pretty sparse on the initial implementation but the idea was that the rows in the results are the columns of the schema, so an additional column of this result could for example be the type of the column. The whole point was that we could do things such as what you can do in postgres with select * from information_schema.columns where table_name = 'stud1';. The problem is though that we don't actually have a global schema so we will always need a layer that aggregates the potentially different sets of schemas, which is exactly what the distinct query plan does.

The idea was that the interface is that everything produces an arrow.Record, and schema scans are like accessing a different (meta-)table, therefore the expressions should not need to know anything special in regards to whether it's a table-scan or a schema-scan.

Yea I understood the intent, it just made more sense to me to actually return a schema, the types can be added to the actual schema (I just skipped that for now). Otherwise you need to know that there is a magic column returned called name.