qpmodel icon indicating copy to clipboard operation
qpmodel copied to clipboard

use execution feedback to improve CE / re-optimization / AQP

Open zhouqingqing opened this issue 4 years ago • 0 comments

The theory is here: https://arxiv.org/abs/1711.08330 and an implementation with PostgreSQL is here: https://github.com/postgrespro/aqo

A related topic is re-optimization (https://arxiv.org/pdf/1902.08291.pdf), where when executor detects discrepancy between optimizer's estimation and actual data, query can kick out reoptimization to fully redo or partially redo the query. It is also called adaptive query execution (in spark's term) where 3 adjustments made:

  • shuffle adjust
  • join method
  • join skew handling

Note that spark's staged execution is easier for this mode because it can clearly cut done|not done parts of the query but mppdb stylestreaming based queries needs more designs (similar to staged query restart idea).

zhouqingqing avatar May 18 '20 01:05 zhouqingqing