SQL Support : Need Aggregation Plan working

[guregodevo]

I'm interested in running SQL queries that requires Selection and Aggregation plans (basic avg, sum and count aggregates). I tried to implement the Aggregation plans and I'm not sure what's your plans and ideas on it. Also the Selection plan seems affordable but I wonder what are the specs. Can we add the selection plan with minimum viable expressions to get it working? I'd be eager to start working on it if you have any open issues and some ideas to share.

[chrislusf]

No plan yet. Maybe help to write up your design first. Doesn't need to be perfect. Let's iterate on this.

I think the tricky part is how to manage the schema.

[guregodevo]

For selection, the strategy would be to push down all predicates when converting to physical Table Scan. I would ignore the index Scan and consider naively it always pushes down to a physical Table Scan. Actually it seems it is already implemented in the convert2PhysicalScan function of the Selection plan.

If I test it with a simple predicate :

golang

sqlText := `
	select line, word as w, line as l2, word is null as x
	from words
	where line = 6
`
f := flow.New("testSelection")

ds := f.Slices([][]interface{}{
	{"this", 1},
	{"is", 2},
	{"a", 3},
	{"table", 4},
	{"that", 5},
	{"are", 6},
	{"many", 7},
	{"pencils", 6},
}).RoundRobin("rr", 2)

sql.RegisterTable(ds, "words", []executor.TableColumn{
	{"word", mysql.TypeVarchar},
	{"line", mysql.TypeLong},
})

out, p, err := sql.Query(sqlText)
if err != nil {
	fmt.Printf("error: %s\n", err)
	return
}

out.Fprintf(os.Stdout, "%v \n")

golang

If I debug I'm seeing the initial logical plan as expected: DataScan(words)->Selection->Projection Then it is converted to physical plan : *plan.PhysicalTableScan->*plan.PhysicalUnionScan->Projection And the bottom PhysicalTableScan has the predicate "line eq 6" as expected.

I'm not sure where i should fix it to make this selection work.

[chrislusf]

Current SQL implementation is half baked and not good. Need to redo it. On Mon, May 21, 2018 at 7:26 AM Gregory Desvaux [email protected] wrote:

For selection, the strategy would be to push down all predicates when converting to physical Table Scan. I would ignore the index Scan and consider naively it always pushes down to a physical Table Scan. Actually it seems it is already implemented in the convert2PhysicalScan function of the Selection plan.

If I test it with a simple predicate :

golang

sqlText := select line, word as w, line as l2, word is null as x from words where line = 6 f := flow.New("testSelection")

ds := f.Slices([][]interface{}{ {"this", 1}, {"is", 2}, {"a", 3}, {"table", 4}, {"that", 5}, {"are", 6}, {"many", 7}, {"pencils", 6}, }).RoundRobin("rr", 2)

sql.RegisterTable(ds, "words", []executor.TableColumn{ {"word", mysql.TypeVarchar}, {"line", mysql.TypeLong}, })

out, p, err := sql.Query(sqlText) if err != nil { fmt.Printf("error: %s\n", err) return }

out.Fprintf(os.Stdout, "%v \n")

golang

If I debug I'm seeing the initial logical plan as expected: DataScan(words)->Selection->Projection Then it is converted to physical plan : *plan.PhysicalTableScan->*plan.PhysicalUnionScan->Projection And the bottom PhysicalTableScan has the predicate "line eq 6" as expected.

I'm not sure where i should fix it to make this selection work.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/chrislusf/gleam/issues/127#issuecomment-390669997, or mute the thread https://github.com/notifications/unsubscribe-auth/ABeL7ycTBkaQ73TNVuHaojObEIvbnFqOks5t0s5_gaJpZM4TuipT .