gpdb icon indicating copy to clipboard operation
gpdb copied to clipboard

[6X] Use full precision for statistics values in Orca

Open chrishajas opened this issue 1 year ago • 0 comments

Previously, if a frequency or ndv value was very small, it could get truncated to 0. In 7X, we bypass this translation logic for queries in the database, but in 6X we'll still do this translation. By translating the full precision, we'll improve cardinality estimates for queries with very small ndv or frequency values.

Note: CFilterCardinalityTest was modified to not apply the filter twice as it introduced flakiness.

This is a backport of a commit that introduced the SetFullPrecision() functionality and a subsequent commit that extends this logic for frequency and ndvs. Initially, I created a PR that bypassed this logic (https://github.com/greenplum-db/gpdb/pull/17374), but to keep changes more minimal we're going with this approach instead. No difference in optimization time was observed with this.

chrishajas avatar Apr 23 '24 22:04 chrishajas