Skip to content

Using dbt_utils.star in dbt model seems to break column lineage in sqlmesh ui #4873

Open
@anentropic

Description

@anentropic

I have a complicated-ish dbt model with several CTEs and incremental

I've been gradually narrowing down what causes column lineage in sqlmesh UI to break with this error:

web.server.exceptions.ApiException: Summary: Unable to get column lineage
Failed to render query for model 'analyticsng_gold_triage.followups'.
Traceback (most recent call last):
  File "~/dev/analytics-sqlmesh/.venv/lib/python3.13/site-packages/web/server/api/endpoints/lineage.py", line 145, in column_lineage
    return create_lineage_adjacency_list(model_name, column_name, context)
  File "~/dev/analytics-sqlmesh/.venv/lib/python3.13/site-packages/web/server/api/endpoints/lineage.py", line 65, in create_lineage_adjacency_list
    root = lineage(quote_column(column, model.dialect), model)
  File "~/dev/analytics-sqlmesh/.venv/lib/python3.13/site-packages/sqlmesh/core/lineage.py", line 39, in lineage
    query = t.cast(exp.Query, model.render_query_or_raise().copy())
                              ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "~/dev/analytics-sqlmesh/.venv/lib/python3.13/site-packages/sqlmesh/core/model/definition.py", line 352, in render_query_or_raise
    raise SQLMeshError(f"Failed to render query for model '{self.name}'.")
sqlmesh.utils.errors.SQLMeshError: Failed to render query for model 'analyticsng_gold_triage.followups'.

and it seems that any use of dbt_utils.star is what causes it (from https://github.com/dbt-labs/dbt-utils)

in the full model if I just replace all the star macro calls with either plain * or paste in the expanded column names from dbt compiled sql then the error goes away and lineage works perfectly

and then if I simplify the model to just:

with source as (
    select
        {{ dbt_utils.star(ref('int_followups')) }}
    from {{ ref('int_followups') }}
)
select *
from source

it breaks, but:

with source as (
    select
        *
    from {{ ref('int_followups') }}
)
select *
from source

...works.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions