timescaledb icon indicating copy to clipboard operation
timescaledb copied to clipboard

1 step policy for continuous aggregates

Open RafiaSabih opened this issue 1 year ago • 4 comments

This simplifies the process of adding the policies for the CAggs. Now, with one single sql statements all the policies can be added for a given CAgg. Similarly, all the policies can be removed or modified via single sql statement only. This also adds a new function as well as a view to show all the policies on a continuous aggregate. While adding or updating a policy it also checks if that policy is compatible with other policies on the CAgg.

Disable-check: commit-count

RafiaSabih avatar Aug 01 '22 16:08 RafiaSabih

@gayyappan, @akuzm: please review this pull request.

Powered by pull-review

github-actions[bot] avatar Aug 01 '22 16:08 github-actions[bot]

Codecov Report

Merging #4563 (90c7e5e) into main (be429eb) will increase coverage by 0.07%. The diff coverage is 96.33%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #4563      +/-   ##
==========================================
+ Coverage   90.08%   90.16%   +0.07%     
==========================================
  Files         223      224       +1     
  Lines       41463    41851     +388     
==========================================
+ Hits        37354    37733     +379     
- Misses       4109     4118       +9     
Impacted Files Coverage Δ
src/ts_catalog/continuous_agg.h 100.00% <ø> (ø)
tsl/src/init.c 95.45% <ø> (ø)
tsl/src/bgw_policy/compression_api.c 85.02% <87.50%> (+1.89%) :arrow_up:
tsl/src/bgw_policy/policies_v2.c 96.38% <96.38%> (ø)
tsl/src/bgw_policy/continuous_aggregate_api.c 93.35% <97.87%> (+0.41%) :arrow_up:
src/cross_module_fn.c 70.44% <100.00%> (+0.95%) :arrow_up:
tsl/src/bgw_policy/retention_api.c 93.93% <100.00%> (+5.05%) :arrow_up:
src/loader/bgw_message_queue.c 85.52% <0.00%> (-2.64%) :arrow_down:
tsl/src/nodes/data_node_dispatch.c 96.49% <0.00%> (-0.24%) :arrow_down:
tsl/src/bgw_policy/job.c 88.62% <0.00%> (-0.05%) :arrow_down:
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update be429eb...90c7e5e. Read the comment docs.

codecov[bot] avatar Aug 01 '22 16:08 codecov[bot]

I've seen 2 problems during testing:

  1. If I create a retention policy with add_retention_policy() on the underlying hypertable the "do not allow refreshed data to be deleted" is not triggered. In any case, we need this test case for coverage reasons.
  2. If I run SELECT * FROM timescaledb_experimental.policies; when I have policies for both the original and materialized hypertables the output is mixed up. There is probably something wrong with the view definition. We should add this test case as well.

mfundul avatar Aug 01 '22 18:08 mfundul

@mfundul, @nikkhils: please review this pull request.

Powered by pull-review

github-actions[bot] avatar Aug 09 '22 09:08 github-actions[bot]