Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug Fix: Coalesce incremental fields #87

Merged
merged 3 commits into from
Aug 27, 2024

Conversation

fivetran-avinash
Copy link
Contributor

@fivetran-avinash fivetran-avinash commented Aug 27, 2024

PR Overview

This PR will address the following Issue/Feature: [#86]

This PR will result in the following new package version: v0.13.1

No field names were changed, just the underlying behavior for null coalesces.

Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:

Bug Fixes

  • Coalesces the backfill_lifetime_sums fields from incremental loads, as well as cohort_month_number in the rare cases there are no orders from an incremental period. This fixes the issue of NULL values in the lifetime columns in shopify__customer_cohorts table. (PR #86).

Under the Hood:

  • Added consistency and integrity tests within integration_tests for the shopify__customer_cohorts model.

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

  • dbt run –full-refresh && dbt test
  • dbt run (if incremental models are present) && dbt test (customer)

Before marking this PR as "ready for review" the following have been applied:

  • The appropriate issue has been linked, tagged, and properly assigned
  • All necessary documentation and version upgrades have been applied
  • docs were regenerated (unless this PR does not include any code or yml updates)
  • BuildKite integration tests are passing
  • Detailed validation steps have been provided below

Detailed Validation

Please share any and all of your validation steps:

Customer validated that the incremental run worked as expected internally.

We created validation tests as well on the end model impacted, and were able to get a successful pass on our integration tests data.

Screenshot 2024-08-27 at 3 28 41 PM

If you had to summarize this PR in an emoji, which would it be?

🏃🏽

advolut-team and others added 2 commits August 26, 2024 11:36
* coalesce incremental load values

* coalesce windows columns
@fivetran-avinash fivetran-avinash self-assigned this Aug 27, 2024
Copy link
Contributor

@fivetran-catfritz fivetran-catfritz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-avinash I saw that the customer tested this change, but should we do our own validation?

Copy link
Contributor

@fivetran-catfritz fivetran-catfritz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm for release!

@fivetran-avinash fivetran-avinash merged commit 470b8da into main Aug 27, 2024
9 checks passed
@fivetran-avinash fivetran-avinash linked an issue Aug 28, 2024 that may be closed by this pull request
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] Incremental Load in shopify__customer_cohorts has NULL fields
3 participants