gpdb
gpdb copied to clipboard
[6X] Use full precision for statistics values in Orca
Previously, if a frequency or ndv value was very small, it could get truncated to 0. In 7X, we bypass this translation logic for queries in the database, but in 6X we'll still do this translation. By translating the full precision, we'll improve cardinality estimates for queries with very small ndv or frequency values.
Note: CFilterCardinalityTest was modified to not apply the filter twice as it introduced flakiness.
This is a backport of a commit that introduced the SetFullPrecision() functionality
and a subsequent commit that extends this logic for frequency and ndvs. Initially, I
created a PR that bypassed this logic (https://github.com/greenplum-db/gpdb/pull/17374),
but to keep changes more minimal we're going with this approach instead. No difference
in optimization time was observed with this.