Skip to content

[Bug] DBT fails to create valid BigQuery SQL code when specifying input data for unit tests in YAML with nested columns #1306

@darkclouder

Description

@darkclouder

Is this a new bug?

  • I believe this is a new bug
  • I have searched the existing issues, and I could not find an existing issue for this bug

Which packages are affected?

  • dbt-adapters
  • dbt-tests-adapter
  • dbt-athena
  • dbt-athena-community
  • dbt-bigquery
  • dbt-postgres
  • dbt-redshift
  • dbt-snowflake
  • dbt-spark

Current Behavior

DBT fails to create valid BigQuery SQL code when specifying input data for unit tests in YAML with nested columns.

unit_tests:
  - name: my_assert
    model: my_model
    given:
      - input: ref('my_table')
        rows:
          - id: 123
            features:
              list:
                - element: door

turns into

select
  safe_cast('''123''' as INTEGER) as id,
  safe_cast({'list': [{'element': 'door'}]} as STRUCT<`list` ARRAY<STRUCT<`element` STRING>>>) as features,

which is invalid BigQuery SQL code:

Braced constructors are not supported at [3:13] 

Expected Behavior

Expected valid SQL code, maybe like

struct([struct('door' as element)] as list)

Steps To Reproduce

  1. Create model my_model.sql
SELECT * FROM {{ ref('my_table') }}
  1. Create unit test yaml file in models folder as mentioned above
  2. Run dbt test --select my_assert

Environment

- OS: Debian 5.10
- Python: 3.10
- dbt-adapters: 1.14.1
- dbt-bigquery: 1.9.1

Additional Context

Seems like some people try to work around it with directly specifying the struct as SQL:
#567

Current workaround:

      - input: ref('my_table')
        format: sql
        rows: |
          select
            123 as id,
            struct([struct('door' as element)] as list) as features

Metadata

Metadata

Assignees

No one assigned

    Labels

    triage:productIn Product's queuetype:bugSomething isn't working as documented

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions