Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature/auto-releaser-bot #4

Merged
merged 2 commits into from
Oct 21, 2024
Merged

Conversation

fivetran-joemarkiewicz
Copy link
Collaborator

PR including the auto releaser and also the trigger for the initial release of this package. See below for the initial release notes.

This is the initial release of the Unified RAG dbt package!

What does this dbt package do?

The main focus of this dbt package is to generate an end model and Cortex Search Service (for Snowflake destinations only) which contains the below relevant unstructured document data to be used for Retrieval Augmented Generation (RAG) applications leveraging Large Language Models (LLMs):

The following table provides a detailed list of all models materialized within this package by default.

TIP: See more details about these models in the package's dbt docs site.

Table Description
rag__unified_document Each record represents a chunk of text prepared for semantic-search and additional fields for use in LLM workflows.

Additionally, for Snowflake destinations, a Cortex Search Service will be generated as a result of this data model. The Cortex Search Service uses the results of the rag__unified_document and enables Snowflake users to take advantage of low-latency, high quality "fuzzy" search over their data for use in RAG applications leveraging LLMs. See the below table for details.

Snowflake Cortex Search Service Description
rag__unified_search Generates a Snowflake Cortex Search service via the search_generation macro as a post-hook for Snowflake destinations. This Cortex Search Service is currently configured with a target lag of 1 day. Please be aware that this search service will refresh automatically once a day even outside of this data model execution. To understand more about the Cortex Search Service, you can run SHOW CORTEX SEARCH SERVICES in the respective Snowflake database.schema which the rag__unified_document is materialized. See here for other relevant commands to use for understanding the nature of the Search Service, and here for helpful commands to use when leveraging the results of the Cortex Search Service in your LLM applications.

@fivetran-avinash fivetran-avinash self-requested a review October 21, 2024 18:30
Copy link
Contributor

@fivetran-avinash fivetran-avinash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@fivetran-joemarkiewicz fivetran-joemarkiewicz merged commit fd581a0 into main Oct 21, 2024
7 checks passed
@fivetran-joemarkiewicz fivetran-joemarkiewicz deleted the feature/auto-releaser-bot branch October 21, 2024 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants