Skip to content

Fix!: Always recreate materialized view #4908

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

VaggelisD
Copy link
Contributor

Materialized views in engines like BigQuery and Snowflake are invalidated if the underlying table(s) are replaced. Toy repro scenario:

1. CREATE TABLE tbl AS (SELECT 1 AS col);

2. CREATE MATERIALIZED VIEW mview AS (SELECT * FROM tbl);

3. CREATE OR REPLACE TABLE tbl AS (SELECT 2 AS col);

4. SELECT * FROM mview;

This flow can be mirrored in the following toy SQLMesh project:

# Full model
MODEL (name example_schema.t1, kind FULL); SELECT 1 AS col;

# Materialized view
MODEL (
  name example_schema.t2,
  kind VIEW (
    materialized true
  )
);

SELECT * FROM example_schema.t1;

Running the very first plan will yield the following error in either engine, e.g:

google.api_core.exceptions.BadRequest: 400 Materialized view <t2> references table <t1> which was deleted and recreated. The view must be deleted and recreated as well

This is because, during the plan:

  1. The physical table for t1 is created as a dry run
  2. The materalized view for t2 is created as a dry run
  3. The physical table for t1 is recreated/replaced as part of the FULL insertion strategy
  4. The materialized view exists, and based on the pre-existing ViewStrategy it is not recreated

This PR alters (4) such that any materialized view with upstream dependencies is recreated from scratch in order to restore correctness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants