promql-engine icon indicating copy to clipboard operation
promql-engine copied to clipboard

plan,execution: late coalesce experiments

Open MichaHoffmann opened this issue 2 years ago • 0 comments

So far only a sketch to get a feeling; benchmark ( if i get rid of batch size! ) looks fairly promising though:

$ go test . -run xxx -bench BenchmarkRangeQuery/experiment/new_engine -v -test.count 5
goos: linux
goarch: amd64
pkg: github.com/thanos-io/promql-engine/engine
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
BenchmarkRangeQuery
BenchmarkRangeQuery/experiment
BenchmarkRangeQuery/experiment/new_engine
BenchmarkRangeQuery/experiment/new_engine-8         	      31	  35560327 ns/op	 8950556 B/op	   70752 allocs/op
BenchmarkRangeQuery/experiment/new_engine-8         	      34	  34607836 ns/op	 8947958 B/op	   70744 allocs/op
BenchmarkRangeQuery/experiment/new_engine-8         	      36	  33039457 ns/op	 8930715 B/op	   70658 allocs/op
BenchmarkRangeQuery/experiment/new_engine-8         	      37	  31686100 ns/op	 8918604 B/op	   70638 allocs/op
BenchmarkRangeQuery/experiment/new_engine-8         	      36	  32301408 ns/op	 8930411 B/op	   70669 allocs/op
PASS
ok  	github.com/thanos-io/promql-engine/engine	17.815s

vs on main

$ go test . -run xxx -bench BenchmarkRangeQuery/experiment/new_engine -v -test.count 5
goos: linux
goarch: amd64
pkg: github.com/thanos-io/promql-engine/engine
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
BenchmarkRangeQuery
BenchmarkRangeQuery/experiment
BenchmarkRangeQuery/experiment/new_engine
BenchmarkRangeQuery/experiment/new_engine-8         	      22	  46970488 ns/op	 6916236 B/op	   39532 allocs/op
BenchmarkRangeQuery/experiment/new_engine-8         	      25	  46018216 ns/op	 6908259 B/op	   39510 allocs/op
BenchmarkRangeQuery/experiment/new_engine-8         	      24	  49707030 ns/op	 7131612 B/op	   39635 allocs/op
BenchmarkRangeQuery/experiment/new_engine-8         	      24	  50225178 ns/op	 6981826 B/op	   39560 allocs/op
BenchmarkRangeQuery/experiment/new_engine-8         	      21	  52848319 ns/op	 7079423 B/op	   39609 allocs/op
PASS
ok  	github.com/thanos-io/promql-engine/engine	16.435s

To get to a point where we can do it we should do a few preparatory PRs:

  • replace all vector selectors by our logical vector selectors
  • introduce logical matrix selector with pushed down function
  • make coalesce a logical node too

MichaHoffmann avatar Dec 22 '23 19:12 MichaHoffmann