flux icon indicating copy to clipboard operation
flux copied to clipboard

Introduce Schema introspection function

Open nathanielc opened this issue 7 years ago • 3 comments

It will be valuable for operations to be able to inspect the table schema on the fly.

We propose adding a standard object type for doing so. The schema object will be as follows:

type schema = {
    colums: []column // list of all columns
    grouped:  []string // list of only the group labels
}
type column = {
   label : string, // column label
   type: string, // name for the column type
   grouped: bool, // whether the column is part of the group key
}

Flux functions can then make use of the schema record for defining dynamic behaviors on the schema

Take for example the group function that wants a list of column labels.

A user could write this to add a column to the list of grouped columns without needing to know the existing group columns

group(by: (schema) => schema.grouped + ["host"])

nathanielc avatar Sep 12 '18 20:09 nathanielc

There's a typo in the schema. It should be columns.

I also think that groupKeys or keys are better names than grouped and that the column boolean attribute should be isKey. Those are just preferences though.

jsternberg avatar Sep 12 '18 21:09 jsternberg

Since the group key segments can be obtained from the column data, maybe we can have the top-level groupKey parameter be a concatenated string? that way two schemas can be quickly compared for equality by the groupKey

edit: didn't see your example with group . But it would be useful to have a quick test to see if two tables have the same group columns.

aanthony1243 avatar Sep 14 '18 14:09 aanthony1243

new question: in #28 we propose passing schema.columns to a filter() function. This would require overloading the existing filter() function, or else having a filterList() function or similar.

aanthony1243 avatar Sep 14 '18 15:09 aanthony1243