cockroach
cockroach copied to clipboard
sql: add SHOW STATISTICS WITH FORECAST
sql/stats: replace eval.Context with tree.CompareContext
Most uses of eval.Context in the sql/stats package can actually be tree.CompareContext instead, so make the replacement.
Release note: None
sql/stats: bump histogram version to 2
In 22.2 as of 963deb8 we support multiple histograms for trigram- indexed strings. Let's bump the histogram version for this change, as we may want to know whether multiple histograms are possible for a given row in system.table_statistics.
(I suspect that during upgrades to 22.2 the 22.1 statistics builder will choke on these statistics, so maybe we should also backport a version check to 22.1.)
Also update avgRefreshTime to work correctly in multiple-histogram cases.
Release note: None
sql/stats: teach histogram.adjustCounts to remove empty buckets
Sometimes when adjusting counts down we end up with empty buckets in the histogram. They don't hurt anything, but they take up some memory (and some brainpower when examining test results). So, teach adjustCounts to remove them.
Release note: None
sql/stats: always use non-nil buckets for empty-table histograms
After 82b5926 I've been using the convention that nil histogram buckets = no histogram, and non-nil-zero-length histogram buckets = histogram on empty table. This is mostly useful for testing but is also important for forecasting histograms.
Fix a spot that wasn't following this convention.
Also, add some empty-table testcases and some other testcases for histogram.adjustCounts.
Release note: None
sql/stats: forecast table statistics
Add function to forecast table statistics based on observed statistics. These forecasts are based on linear regression models over time. For each set of columns with statistics, we construct a linear regression model over time for each statistic (row count, null count, distinct count, average row size, and histogram). If all models are good fits then we produce a statistics forecast for the set of columns.
Assists: #79872
Release note: None
sql: add SHOW STATISTICS WITH FORECAST
Add a new WITH FORECAST option to SHOW STATISTICS which calculates and displays forecasted statistics along with the existing table statistics.
Also, forbid injecting forecasted stats.
Assists: #79872
Release note (sql change): Add a new WITH FORECAST option to SHOW STATISTICS which calculates and displays forecasted statistics along with the existing table statistics.
This is now compiling and not crashing on some example statistics, so I wanted to open a draft PR as a sneak preview. It still needs tests before it is ready for review.
PRs after this will wire the forecasts into the stats_cache.
Added some tests, fixed some bugs. A few tests still need to be written.
This is RFAL. I need to write two more tests, but that can happen concurrently with reviewing.
Make some linters happy. 🙂
Rebased on the previous patches, but haven't yet addressed feedback. Hold off on reviewing just yet.
TFYRs!!
bors r+
:-1: Rejected by code reviews
bors r+