Skip to content

union_relations: passing include parameter makes column_override fields case sensitive in data types quietly #1017

Open
@ttusing

Description

@ttusing

Describe the bug

When using the union_relations macro, when passing an include parameter, column_override requires the passed field names to be uppercase. Otherwise, the data types are not picked up for casting and the default behavior of using the first relation's datatypes are used.

Usability note/alternative: It's not clear to me if/why one should use both include and column_override, would be nice to just provide a dictionary with data types like column_override and know I also get the include behavior.

Steps to reproduce

  • Make a model using dbt_utils.union_relations to union two relations where not all upstream datatypes are varchar
  • Add an include list of all fields or other set
  • Add a column_override list that would cast a datatype in lowercase (or mixed case) SQL, like {"id": "number(38,0)", ...}
  • Run DBT build on the model
  • Observe the datatypes in the output

Expected results

column_override datatypes are used for casting regardless of include list

The interaction between include and union_relations is clear and documented. At a minimum, providing an include list should not make union_relations case sensitive.

Actual results

column_override is case sensitive and silently ignored when include list is passed.

Screenshots and log output

System information

The contents of your packages.yml file:

Which database are you using dbt with?

  • postgres
  • redshift
  • bigquery
  • snowflake
  • other (specify: ____________)

The output of dbt --version:

1.9.4

Additional context

Are you interested in contributing the fix?

Yes!

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtriage

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions