Skip to content

Add workaround for downgrade in 2.22.0 #48

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Add workaround for downgrade in 2.22.0 #48

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ad11b80
Add workaround for downgrade in 2.22.0
kpan2034 Sep 8, 2025
bfe912a
Fix to_timestamp function usage
kpan2034 Sep 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions utils/2.22.0-downgrade-cagg-materialization-ranges/01-save-materialization-ranges.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
\set ON_ERROR_STOP ON

SELECT *
INTO _timescaledb_internal.saved_ranges
Copy link
Member

@fabriziomello fabriziomello Sep 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also maybe continuous_aggs_materialization_ranges_before_downgrade_2_22_0 can be a better name to also express the intention.

I think it should be saved it in the public schema instead cause is preferred to let the extension update/downgrade script mess with our internal schemas.

FROM _timescaledb_catalog.continuous_aggs_materialization_ranges;
Comment on lines +3 to +5
Copy link
Member

@fabriziomello fabriziomello Sep 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
SELECT *
INTO _timescaledb_internal.saved_ranges
FROM _timescaledb_catalog.continuous_aggs_materialization_ranges;
CREATE TABLE _timescaledb_internal.saved_ranges AS
SELECT *
FROM _timescaledb_catalog.continuous_aggs_materialization_ranges
ORDER BY materialization_id, lowest_modified_value;

Nit. IMHO using CREATE TABLE instead of SELECT INTO express better the intention.


TRUNCATE _timescaledb_catalog.continuous_aggs_materialization_ranges;

\set ON_ERROR_STOP OFF
21 changes: 21 additions & 0 deletions utils/2.22.0-downgrade-cagg-materialization-ranges/02-refresh-saved-ranges.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
\set ON_ERROR_STOP 1

DO $$
DECLARE
range_record RECORD;
BEGIN
Copy link
Member

@fabriziomello fabriziomello Sep 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe here you can check for the existance of the stating table and raise an exception that the table don't exists so probably the script was already executed.

FOR range_record IN
SELECT sr.materialization_id, sr.lowest_modified_value AS lowest, sr.greatest_modified_value AS greatest,
ca.user_view_schema || '.' || ca.user_view_name AS cagg_name
FROM _timescaledb_internal.saved_ranges sr
JOIN _timescaledb_catalog.continuous_agg ca ON sr.materialization_id = ca.mat_hypertable_id
ORDER BY sr.materialization_id
LOOP
RAISE NOTICE 'force refreshing %s: start:%s end:%s', range_record.cagg_name, _timescaledb_functions.to_timestamp(range_record.lowest), _timescaledb_functions.to_timestamp(range_record.greatest);
CALL refresh_continuous_aggregate(range_record.cagg_name, _timescaledb_functions.to_timestamp(range_record.lowest), _timescaledb_functions.to_timestamp(range_record.greatest), force => true);
Copy link
Member

@fabriziomello fabriziomello Sep 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should also consider integer time dimension

END LOOP;
END $$;

DROP TABLE _timescaledb_internal.saved_ranges;
Comment on lines +16 to +19
Copy link
Member

@fabriziomello fabriziomello Sep 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
END LOOP;
END $$;
DROP TABLE _timescaledb_internal.saved_ranges;
END LOOP;
DROP TABLE _timescaledb_internal.saved_ranges;
END $$;

I think it must be part of the anonymous transaction.


\set ON_ERROR_STOP 0
38 changes: 38 additions & 0 deletions utils/2.22.0-downgrade-cagg-materialization-ranges/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Issue downgrading from 2.22.0 due to unrefreshed CAggs

In 2.22.0 we introduced an additional transaction during the CAgg refresh which processes ranges from a new table `_timescaledb_catalog.continuous_aggs_materialization_ranges`.
This staging table was introduced in order to help fix issues relating to concurrent CAgg refreshes: [#8372](https://github.com/timescale/timescaledb/issues/8372), [#8490](https://github.com/timescale/timescaledb/issues/8490)

Since this table would be dropped on a downgrade to 2.21.3, we block the downgrade if there are any ranges present in the table, in order to ensure no data loss in dependent CAggs. It is generally expected that this staging table will only contain ranges which are being refreshed.
If there *is* any data leftover in the table, that *isn't* being processed, it is because of an interrupted CAgg refresh, so we ask the user to perform a force refresh of these ranges before proceeding with the downgrade. In most cases, this should suffice.

However, if a force refresh of all CAggs is not a possible option, we are blocked from downgrading completely. This script provides a workaround.

**We recommend trying to force refresh all CAggs before performing the downgrade normally. This script is a workaround which serves only as a backup option.**

## How it works

The workaround is straightforward: we save a copy of the ranges in `_timescaledb_catalog.continuous_aggs_materialization_ranges` in a new table `_timescaledb_internal.saved_ranges`. We then perform the downgrade, which should go through. After this, we force-refresh CAggs using the saved ranges then drop the `saved_ranges` table.

## Usage

Start a `psql` session and do the following:

1. Copy the ranges into a new table.

```SQL
\i 01-save-materialization-ranges.sql
```


2. Open a new psql session and downgrade to timescaledb version 2.21.3:

```SQL
ALTER EXTENSION timescaledb UPDATE TO '2.21.3';
```

3. Force refresh CAggs using the saved ranges and drop the copy:

```SQL
\i 02-refresh-saved-ranges.sql
```