Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Update comment_markdown fields with coalesces so that Snowflake concatenation returns not null elements #20

Open
2 of 4 tasks
fivetran-avinash opened this issue Jan 29, 2025 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@fivetran-avinash
Copy link
Contributor

fivetran-avinash commented Jan 29, 2025

Is there an existing issue for this?

  • I have searched the existing issues

Describe the issue

Snowflake does not support concatenation on null elements. If any fields are being concatenated that are null, the resulting output of the concatenation is null. We will need to perform coalesce on all fields being concatenated so that it always returns a not null value.

Relevant error log or model output

The relevant code being impacted lies in our intermediate models, where the dbt.concat function is applied, say int_rag_zendesk__ticket_comment_document.

       cast(
            {{ dbt.concat([
                "'### message from '", "commenter_name", "' ('", "commenter_email", "')\\n'",
                "'##### sent @ '", "comment_time", "'\\n'",
                "comment_body"
            ]) }} as {{ dbt.type_string() }})
            as comment_markdown

If any one of these fields is null, say commenter_email, or commenter_name, then the output of the comment_markdown will also be null in Snowflake, regardless of any other not-null elements present.

Expected behavior

When concatenated, the output of comment_markdown is any concatenation with not null elements are populated.

Possible solution

We apply our coalesce_cast macro to all fields that are concatenated and create a not-null standardized value to be safe.

dbt Project configurations

Standard

Package versions

packages:

  • package: fivetran/fivetran_utils
    version: [">=0.4.0", "<0.5.0"]

What database are you using dbt with?

snowflake

How are you running this dbt package?

dbt Core™

dbt Version

1.7.4

Additional Context

No response

Are you willing to open a PR to help address this issue?

  • Yes.
  • Yes, but I will need assistance.
  • No.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant