Move invalidations on refresh only if no policy #8191
Conversation
If no move policy is defined, a continuous aggregate refresh will not move invalidations (not even try to move invalidations) and assumes that the process hypertable invalidation policies fully decide how to process hypertable invalidations. If you provide the `force` option to the refresh policy, it is assumed that there are hypertable invalidations that need to be processed and the processing is done.
| /* See if there is a move hypertable invalidations policy defined */ | ||
| List *jobs = | ||
| ts_bgw_job_find_by_proc_and_hypertable_id(POLICY_PROCESS_HYPER_INVAL_PROC_NAME, | ||
| FUNCTIONS_SCHEMA_NAME, | ||
| policy_data.cagg->data.raw_hypertable_id); | ||
| bool has_move_policy = list_length(jobs) > 0; |
There was a problem hiding this comment.
We might want to add a GUC to allow this to be configured so that the default behavior is to always process the hypertable invalidations but it can be enabled.
There was a problem hiding this comment.
Maybe we can use the existing force option
There was a problem hiding this comment.
Yes, this is what the code currently does, but that means you have to pass it explicitly with each call.
I am thinking that you might want to set it up in one of three different ways:
- Always do an invalidations move when running a refresh
- Never do an invalidation move when running a refresh
- Do an invalidation move when there is no move policy defined.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #8191 +/- ##
==========================================
+ Coverage 82.29% 82.38% +0.08%
==========================================
Files 256 256
Lines 47934 47915 -19
Branches 12077 12075 -2
==========================================
+ Hits 39448 39474 +26
- Misses 3643 3654 +11
+ Partials 4843 4787 -56 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| if (!force) | ||
| { | ||
| List *jobs = ts_bgw_job_find_by_proc_and_hypertable_id(POLICY_PROCESS_HYPER_INVAL_PROC_NAME, | ||
| FUNCTIONS_SCHEMA_NAME, | ||
| cagg->data.raw_hypertable_id); | ||
| move_hypertable_invalidations = list_length(jobs) == 0; | ||
| } |
There was a problem hiding this comment.
We might want to have a separate option so that you can force-refresh a range but not process hypertable invalidations.
There was a problem hiding this comment.
I think for now a manual refresh should always move the invalidations.
If no move policy is defined, a continuous aggregate refresh will not move invalidations (not even try to move invalidations) and assumes that the process hypertable invalidation policies fully decide how to process hypertable invalidations.
If you provide the
forceoption to the refresh policy, it is assumed that there are hypertable invalidations that need to be processed and the processing is done.