readme

Author: feikesteenbergen

utils/cagg_refresh_producer_consumer/README.md

@@ -0,0 +1,60 @@
+# Continous Aggregates, incremental parallel setup
+
+This code is exploring the possibilities to do incremental CAgg refreshes in parallel.
+The setup it uses is as following:
+
+## Producer
+
+We have a producer procedure (`schedule_refresh_continuous_aggregate_incremental`),
+which schedules tasks to be picked up by the consumers.
+
+The configuration for this call contains the following keys:
+
+```json
+{
+    "end_offset": "similar to end-offset in the policy",
+    "start_offset": "similar to start-offset in the policy",
+    "continuous_aggregate": "regclass / fully qualified name of the user view for the CAgg",
+    "increment_size": "the size of each individual task, default: chunk_interval",
+    "priority": "priority for these tasks. Lower numbers get processed earlier, default: 100"
+}
+```
+
+Some example calls:
+
+### Producer Examples
+
+#### Schedule multiple jobs for this cagg, with increments of 1 week
+
+```json
+{
+    "end_offset": "14 days",
+    "start_offset": "2 years",
+    "continuous_aggregate": "public.stats_five_minutes",
+    "increment_size": "1 week"
+}
+```
+
+```sql
+CALL _timescaledb_additional.schedule_refresh_continuous_aggregate_incremental(
+    job_id => null,
+    config => $1
+);
+
+#### Schedule jobs for this cagg, for recent data with high priority
+
+```json
+{
+    "end_offset": "7 days",
+    "start_offset": "2 years",
+    "continuous_aggregate": "public.stats_five_minutes",
+    "increment_size": "1 week",
+    "priority": 1
+}
+```
+
+```sql
+CALL _timescaledb_additional.schedule_refresh_continuous_aggregate_incremental(
+    job_id => null,
+    config => $1
+);