Skip to content

ttl not added to local table in distributed_incremental materialization #321

@dl-github-reporter

Description

@dl-github-reporter

Describe the bug

Tried to materialize table with distributed_incremental materialization.
TTL setting doesn't exist for local table.

Steps to reproduce

  1. Add model test_model with config. Run dbt build
    {{ config(
     materialized='distributed_incremental',
     engine='ReplicatedReplacingMergeTree(datetime)',
     incremental_strategy='append',
     order_by='datetime, object_id',
     partition_by='toYYYYMM(datetime)',
     ttl='toStartOfYear(datetime + toIntervalYear(5))',
     sharding_key='cityHash64(object_id)'
    ) }}
     
    SELECT * FROM {{ source('default', 'stg_test_table') }}
  2. Check local table with
SHOW CREATE default.test_model_local
  1. Engine will be
ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{uuid}/{shard}', '{replica}', datetime)
PARTITION BY toYYYYMM(datetime)
ORDER BY (datetime, object_id)
SETTINGS index_granularity = 8192

Expected behaviour

Engine of local table should contain TTL setting

ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{uuid}/{shard}', '{replica}', datetime)
PARTITION BY toYYYYMM(datetime)
ORDER BY (datetime, object_id)
TTL toStartOfYear(datetime + toIntervalYear(5))
SETTINGS index_granularity = 8192

Code examples, such as models or profile settings

version: 2

sources:
  - name: default
    tables:
      - name: stg_test_table

dbt and/or ClickHouse server logs

Configuration

Environment

  • dbt version: 1.7.15
  • dbt-clickhouse version: 1.7.7
  • clickhouse-driver version (if using native): 0.2.6
  • clickhouse-connect version (if using http): 0.7.12
  • Python version: 3.10
  • Operating system: python:3.10-slim docker image

ClickHouse server

  • ClickHouse Server version: 24.3.4.147
  • CREATE TABLE statements for tables involved:
CREATE TABLE default.stg_test_table ON CLUSTER '{cluster}'
(
  object_id String,
  datetime DateTime,
  field1 String,
  field2 String
)
ENGINE = ReplicatedMergeTree('/clickhouse/tables/{uuid}/{shard}', '{replica}')
ORDER BY object_id;

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinggood first issueGood for newcomers

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions