Add column mask support for tables and incremental

[ericj-db]

Resolves #670

Description

• Introduce a new column-level config: column_mask
• Supported for tables and incremental. Throws compiler error for views and materialized views

New config example

models:
  - name: column_mask_sql
    columns:
        - name: id
        - name: password
          column_mask:
            function: mask_function
            using_columns: "id, 'hello', 5"

Executed SQL:

ALTER TABLE column_mask_sql ALTER COLUMN password SET MASK mask_function USING COLUMNS(id, 'hello', 5)

Note that id is a reference to the column id of the same table. 'hello' is a string literal

Limitations

When executing the following steps for incremental materializations, I ran into issues

1. Create a function: mask_function(value STRING)
2. Create a model using this function to mask a column
3. Run the model
4. Update the function: mask_function(value STRING, another_value STRING)
5. Update the model, now specifying using_columns to pass in the extra argument
6. Run the model

The above will fail because applying the changeset (i.e. executing the ALTER TABLE statement to update the mask) is done later than the first SELECT statement that runs against the table. This results in an error like

[WRONG_NUM_ARGS.WITHOUT_SUGGESTION] The `catalog`.`schema`.`mask_function` requires 2 parameters but the actual number is 1.

Looking at the macros, handling this edge case seems like it will need significant refactors. Not sure if it is worth the effort, so I have instead added a runtime error that is thrown when detecting that an existing function's signature has changed

UPDATE: we will not try to do anything fancy to handle this edge case. We consider this a user error as changing the function signature puts the table in a broken state. We will document this clearly in the doc update

Checklist

• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• I have updated the CHANGELOG.md and added information about my change to the "dbt-databricks next" section.

[ericj-db]

Since this is not performed as part of the same transaction as the table creation, I believe there could be a period of time where the masks are not applied. Based on https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-column-mask, it is technically possible to apply masks at table creation

If this is preferred, I could make that change

[benc-db]

Yes, on create we should apply at create time. That's the reason we are restricting the feature to Mat v2, because in v1, the presence of 'select' prevents us from doing column stuff.

[ericj-db]

Addressed this in d62217f

[benc-db]

Looking at the macros, handling this edge case seems like it will need significant refactors. Not sure if it is worth the effort, so I have instead added a runtime error that is thrown when detecting that an existing function's signature has changed

Let's discuss this in our 1:1 so I can understand it better.