Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Running dbt compile can overwrite seed files #10886

Open
2 tasks done
Myles1 opened this issue Oct 18, 2024 · 3 comments · May be fixed by #10887
Open
2 tasks done

[Bug] Running dbt compile can overwrite seed files #10886

Myles1 opened this issue Oct 18, 2024 · 3 comments · May be fixed by #10887
Labels
bug Something isn't working seeds Issues related to dbt's seed functionality

Comments

@Myles1
Copy link

Myles1 commented Oct 18, 2024

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

Running dbt compile can overwrite source files with the newly compiled sql

Expected Behavior

Source files will not be changed

Steps To Reproduce

  1. Create a new dbt environment
  2. Create a dbt_project.yml in another directory
  3. Edit the paths in dbt_project.yml to point to the absolute paths in your dbt project
  4. Run dbt compile

Relevant log output

No response

Environment

- OS:MacOS 14.5 (23F79)
- Python:3.11.7
- dbt:1.9.0-b2

Which database adapter are you using with dbt?

bigquery

Additional Context

It looks like I'm not the first to experience this (https://discourse.getdbt.com/t/seeds-overwrites-csv-file-if-outside-directory-containing-yml-files/12972) but I couldn't find a fix anywhere

@Myles1 Myles1 added bug Something isn't working triage labels Oct 18, 2024
@Myles1 Myles1 linked a pull request Oct 18, 2024 that will close this issue
5 tasks
@dbeatty10 dbeatty10 added the seeds Issues related to dbt's seed functionality label Oct 23, 2024
@dbeatty10
Copy link
Contributor

@Myles1 Thanks for reporting this, linking to relevant thread in Discourse, and opening a PR (#10887) 🏆

I was able to reproduce this using the project files and commands below.

Reprex

dbt_project.yml

# other dbt_project.yml content here

seed-paths: ["/full/absolute/path/to/dbt/project/seeds"]

seeds/my_seed.csv

id
1

Commands:

dbt seed
dbt seed

The first command succeeded for me, but the 2nd didn't because the seed file had been overwritten. The file at seeds/my_seed.csv should have been unchanged.

Instead, it overwrote the file which led to an error like this for the 2nd command:

01:21:22    Compilation Error in seed my_seed (/full/path/seeds/my_seed.csv)
  Row 7 has 4 values, but Table only has 1 columns.
  
  > in macro materialization_seed_default (macros/materializations/seeds/seed.sql)
  > called by seed my_seed (/full/path/seeds/my_seed.csv)

@dbeatty10 dbeatty10 removed the triage label Oct 23, 2024
@dbeatty10 dbeatty10 changed the title [Bug] Running dbt compile can overwrite source files [Bug] Running dbt compile can overwrite seed files Oct 23, 2024
@dbeatty10
Copy link
Contributor

@Myles1 I'm curious if you got this error when running dbt compile, because I could only trigger it when I ran dbt seed or dbt build (along with an absolute path for seed-paths)?

@Myles1
Copy link
Author

Myles1 commented Oct 23, 2024

@Myles1 I'm curious if you got this error when running dbt compile, because I could only trigger it when I ran dbt seed or dbt build (along with an absolute path for seed-paths)?

I was able to get dbt compile to overwrite source files by using an absolute path to models

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working seeds Issues related to dbt's seed functionality
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants