Skip to content

v0.12.0 dbt_shopify
Compare
Choose a tag to compare
@fivetran-data-model-bot fivetran-data-model-bot released this 01 May 19:02
· 38 commits to main since this release
v0.12.0
fc20889
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

PR #76 includes the following updates:

🚨 Breaking Changes 🚨

⚠️ Since the following changes are breaking, a --full-refresh after upgrading will be required.

  • Performance improvements:
    • Added an incremental strategy for the following models. These models were picked for incremental materialization based on the size of their upstream sources.
      • shopify__customer_cohorts (For Databricks SQL Warehouse destinations, this model is materialized as a table without support for incremental runs at this time.)
      • shopify__customer_email_cohorts (For Databricks SQL Warehouse destinations, this model is materialized as a table without support for incremental runs at this time.)
      • shopify__discounts
      • shopify__order_lines
      • shopify__orders
      • shopify__transactions
    • Updated the materialization of shopify__orders__order_line_aggregates to a table. This model draws on several large upstream sources and is also referenced in several downstream models, so this was done to improve performance. This model was not selected for incremental materialization since its structure was not conducive to incremental strategy.
  • To reduce storage, updated the default materialization of the upstream staging models from tables to views. (See the dbt_shopify_source CHANGELOG for more details.)

Features

  • Added a default 7-day look-back to incremental models to accommodate late arriving records. The number of days can be changed by setting the var lookback_window in your dbt_project.yml. See the Lookback Window section of the README for more details.
  • Added macro shopify_lookback to streamline the lookback calculation.
  • Updated the partitioning logic in window functions to use only the necessary columns, depending on whether the unioning feature is used. This benefits mainly Redshift destinations, which can see errors when the staging models are materialized as views.

🪲 Bug Fixes 🪛

  • Corrected the fixed_amount_discount_amount logic to appropriately bring in fixed amount discounts in shopify__orders. PR #78
  • Removed the index=1 filter in stg_shopify__order_discount_code in the dbt_shopify_source package to ensure all discount codes are brought in for every orders. For customers with multiple discount codes in an order, this could update the count_discount_codes_applied field in the shopify__orders and shopify__daily_shop models. PR #78

Under the Hood

  • Updated the maintainer PR template to the current format.
  • Added integration testing pipeline for Databricks SQL Warehouse.
  • Added macro shopify_is_databricks_sql_warehouse for detecting if a Databricks target is an All Purpose Cluster or a SQL Warehouse.

Full Changelog: v0.11.0...v0.12.0

Assets 2
Loading