PR #81 includes the following breaking changes:
- Dispatch update for dbt-utils to dbt-core cross-db macros migration. Specifically
{{ dbt_utils.<macro> }}
have been updated to{{ dbt.<macro> }}
for the below macros:any_value
bool_or
cast_bool_to_text
concat
date_trunc
dateadd
datediff
escape_single_quotes
except
hash
intersect
last_day
length
listagg
position
replace
right
safe_cast
split_part
string_literal
type_bigint
type_float
type_int
type_numeric
type_string
type_timestamp
array_append
array_concat
array_construct
- For
current_timestamp
andcurrent_timestamp_in_utc
macros, the dispatch AND the macro names have been updated to the below, respectively:dbt.current_timestamp_backcompat
dbt.current_timestamp_in_utc_backcompat
dbt_utils.surrogate_key
has also been updated todbt_utils.generate_surrogate_key
. Since the method for creating surrogate keys differ, we suggest all users do afull-refresh
for the most accurate data. For more information, please refer to dbt-utils release notes for this update.- Dependencies on
fivetran/fivetran_utils
have been upgraded, previously[">=0.3.0", "<0.4.0"]
now[">=0.4.0", "<0.5.0"]
.
- If doing a dbt_compile prior to dbt_run, it fails at
int_zendesk__calendar_spine
because the staging model it references is not built yet. This PR changes the intermediate models to reference source tables instead of staging models. (#79)
🚨 This includes Breaking Changes! 🚨
- Databricks compatibility 🧱 (#74).
- Updated README documentation updates for easier navigation and setup of the dbt package (#73).
- Added
zendesk_[source_table_name]_identifier
variables to allow for easier flexibility of the package to refer to source tables with different names (#73). - By default, this package now builds the Zendesk staging models within a schema titled (
<target_schema>
+_zendesk_source
) in your target database. This was previously<target_schema>
+_zendesk_staging
, but we have changed it to maintain consistency with our other packges. See the README for instructions on how to configure the build schema differently.
- Swapped references to the
fivetran_utils.timestamp_diff
macro withdbt_utils.datediff
macro. The dbt-utils macro previously did not support Redshift.
- Quick fix on missing logic in the case statement for determining multi-touch resolution metrics.
- @tonytusharjr (#7).
- This Zendesk Source package now allows for custom fields to be added to the
stg_zendesk__ticket
model. These custom fields will also persist downstream to thezendesk__ticket_enriched
andzendesk__ticket_metrics
models. You may now add your own customer fields to these models by leveraging thezendesk__ticket_passthrough_columns
variable. (#70)
- It was brought to our attention that the
dbt_utils.date_trunc
macro only leverages the default arguments of the date_trunc function in the various warehouses. For example,date_trunc
in Snowflake for theweek
argument produces the starting Monday, while BigQuery produces the starting Sunday. For this package, we want to leverage the start of the week as Sunday. Therefore, logic within the business metric intermediate models has been adjusted to capture the start of the week as Sunday. This was done by leveraging theweek_start
macro within thedbt-date
package. (#68)
- The
0.7.1
release of the zendesk package introduced a bug within thezendesk__sla_policy
model that caused duplicate sla records via a join condition. This join condition has been modified to leverage the more accuratesla_policy_applied.valid_starting_at
field instead of thesla_policy_applied.sla_applied_at
which changes forfirst_reply_time
slas. (#67)
- The logic used to generate the
zendesk__ticket_backlog
model was updated to more accurately map backlog changes to tickets. As the underlyingzendesk__ticket_field_history
model is incremental, we recommend a--full-refresh
after installing this latest version of the package. (#61)
- Addition of the DECISIONLOG.md. This file contains detailed explanations for the opinionated transformation logic found within this dbt package. (#59)
- Added logic required to account for the
first_reply_time
when the first commenter is an internal comment and there are no previous external comments applied to the ticket. (#59) - For those using schedules, incorporates Daylight Savings Time to use the proper timezone offsets for calculating UTC timestamps. Business minute metrics are more accurately calculated, as previously the package did not acknowledge daylight time and only used the standard time offsets (#62).
- Updated the incremental logic within
int_zendesk__field_history_scd
to include an additional partition forticket_id
. This allows for a more accurate generation of ticket backlog records. (#61) - Corrected the spelling of the partition field within the cte in
int_zendesk__field_history_scd
to bepartition
opposed topatition
. (#61)
🎉 dbt v1.0.0 Compatibility Pre Release 🎉 An official dbt v1.0.0 compatible version of the package will be released once existing feature/bug PRs are merged.
- Adjusts the
require-dbt-version
to now be within the range [">=1.0.0", "<2.0.0"]. Additionally, the package has been updated for dbt v1.0.0 compatibility. If you are using a dbt version <1.0.0, you will need to upgrade in order to leverage the latest version of the package.- For help upgrading your package, I recommend reviewing this GitHub repo's Release Notes on what changes have been implemented since your last upgrade.
- For help upgrading your dbt project to dbt v1.0.0, I recommend reviewing dbt-labs upgrading to 1.0.0 docs for more details on what changes must be made.
- Upgrades the package dependency to refer to the latest
dbt_zendesk_source
. Additionally, the latestdbt_zendesk_source
package has a dependency on the latestdbt_fivetran_utils
. Further, the latestdbt_fivetran_utils
package also has a dependency ondbt_utils
[">=0.8.0", "<0.9.0"].- Please note, if you are installing a version of
dbt_utils
in yourpackages.yml
that is not in the range above then you will encounter a package dependency error.
- Please note, if you are installing a version of
-
Updated logic within
int_zendesk__sla_policy_applied
to more accurately reflect thesla_applied_at
time forfirst_reply_time
sla's. Per Zendesk's documentation thefirst_reply_time
sla is set at the creation of the ticket, even if the sla is applied after creation. (#52) -
It was found that
first_reply_time
Zendesk SLA policies can be modified after they are set if the priority of the ticket changes. As such, this resulted in the package providing multiplefirst_reply_time
sla records in the finalzendesk__sla_policies
output model. As such, now only the latestfirst_reply_time
sla is provided in the final output model. (#52)
- Redshift recently included
pivot
as a reserved word within the warehouse. As such, thepivot
CTE within theint_zendesk__field_history_pivot
model has been changed topivots
to avoid the Redshift error. (#57)
- @jackiexsun (#52)
- Fix incremental logic bug introduced in v0.5.0 which caused the
zendesk__ticket_field_history
model to not be properly incrementally updated. (#44)- The above fix resulted in the removal of the
valid_from
andvalid_to
fields in the final model.
- The above fix resulted in the removal of the
- Incremental bug fix noted in the
Breaking Changes
section of the changelog. - Updated the logic used to calculate
first_reply_time_calendar_minutes
andfirst_reply_time_business_minutes
to include first comments made by agents and find the time difference from the first public agent and the ticket created date. This was updated to better align with Zendesk's First Reply Time metric definition. (#50) - Fixed the comment metric reference for the
total_agent_replies
withinzendesk__ticket_metrics
to accurately map to thecount_agent_comments
metric (showing all public and non-public comments made by agents) opposed to thecount_internal_comments
(only non-public comments) metric. (#50)
- Add the number of ticket handoffs metric as
count_ticket_handoffs
to thezendesk__ticket_metrics
model which is a distinct count of all internal users who have touched/commented on the ticket. (#42) - Ticket field history calendar limit variables (#47):
- Added
ticket_field_history_timeframe_years
variable to limit the ticket field history model to X number of years (default is 50). - Limited by default the last ticket calendar date as it's close date. This highly reduces the query cost of the
zendesk__ticket_field_history
query and takes advantage of the Zendesk functionality of not being able to change a ticket after the close date. - Added
ticket_field_history_extension_months
variable to extend field history past Y months from ticket close (for reporting purposes). - Refer to the README for more details.
- Added
- Better Postgres incremental strategy within the
zendesk__ticket_field_history
model to reflect more recent incremental strategies. Similar to the strategy taken in jira__daily_issue_field_history. (#44)
- csaroff (#47)
- jackiexsun (#42)
- emiliedecherney (#50)
- gareginordyan (#44)