Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug Fix] Bring back fivetran_deleted for gl_detail models #20

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

[Bug Fix] Bring back fivetran_deleted for gl_detail models #20

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d487b82
Bring back fivetran_deleted
fivetran-avinash Feb 1, 2025
1da9f58
schema
fivetran-avinash Feb 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ If you are **not** using the [Sage Intacct transformation package](https://githu
```yaml
packages:
- package: fivetran/sage_intacct_source
version: [">=0.3.0", "<0.4.0"] # we recommend using ranges to capture non-breaking changes automatically
version: [">=0.4.0", "<0.5.0"] # we recommend using ranges to capture non-breaking changes automatically
```
### Step 3: Define database and schema variables
By default, this package runs using your destination and the `sage_intacct` schema. If this is not where your Sage Intacct data is (for example, if your Sage Intacct schema is named `sage_intacct_fivetran`), add the following configuration to your root `dbt_project.yml` file:
Expand Down
2 changes: 1 addition & 1 deletion dbt_project.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 'sage_intacct_source'
version: '0.3.1'
version: '0.4.0'
config-version: 2
require-dbt-version: [">=1.3.0", "<2.0.0"]
models:
Expand Down
4 changes: 3 additions & 1 deletion integration_tests/dbt_project.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 'sage_intacct_source_integration_tests'
version: '0.3.1'
version: '0.4.0'
config-version: 2

profile: 'integration_tests'
Expand All @@ -15,6 +15,8 @@ vars:
sage_intacct_ar_invoice_item_identifier: "sage_intacct__ar_invoice_item"
sage_intacct_schema: sage_intacct_source_integration_tests_2

# sage_intacct_schema: zz_dbt_avinash_dev

seeds:
+quote_columns: "{{ true if target.type == 'redshift' else false }}"

Expand Down
22 changes: 11 additions & 11 deletions integration_tests/seeds/sage_intacct__gl_detail.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
recordno,accountno,accounttitle,amount,batch_date,batch_no,batch_title,batchkey,bookid,creditamount,debitamount,customerid,customername,description,docnumber,entry_date,entry_state,entrydescription,line_no,recordid,recordtype,totaldue,totalentered,totalpaid,tr_type,trx_amount,trx_creditamount,trx_debitamount,vendorid,vendorname,whencreated,whendue,whenmodified,whenpaid
YGy2nQ7YHphQv5+5AdQTyQ==,15030,A+5CoJT/HDmjwrGYGL6RUg==,2492.5,2021-12-01,74885,El+ECK1x7a25SuLqaMwODA==,506019,/8yT7c/TpIKmxe0K8MmSNA==,0,2492.5,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,06m0www4a75rDMeUgh3o0w==,2021-12-01,P,06m0www4a75rDMeUgh3o0w==,3,20760,1lkow/+J96atpaza+bjZwQ==,2492.5,2492.5,0,1,2492.5,0,2492.5,V-005638,NJa+iTk8TGFrCNzU4GaQMg==,2021-11-16,2021-11-16,2021-12-10 11:47:03,2021-12-10
GA/oILgpDw/6ay94J1jgLA==,20000,HtcR/HtKU0hM5hLuZcjjuA==,-2492.5,2021-12-01,74885,El+ECK1x7a25SuLqaMwODA==,506019,/8yT7c/TpIKmxe0K8MmSNA==,2492.5,0,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,06m0www4a75rDMeUgh3o0w==,2021-12-01,P,1B2M2Y8AsgTpgAmY7PhCfg==,2,20760,1lkow/+J96atpaza+bjZwQ==,2492.5,2492.5,0,-1,-2492.5,2492.5,0,V-005638,NJa+iTk8TGFrCNzU4GaQMg==,2021-11-16,2021-11-16,2021-12-08 16:00:57,2021-12-10
aA1UkdHgMiUM9APkcv3MXA==,20000,HtcR/HtKU0hM5hLuZcjjuA==,-6825,2021-12-01,75155,SNfyN9QHUE4npoS72Hd1Bw==,506509,/8yT7c/TpIKmxe0K8MmSNA==,6825,0,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,2021-12-01,P,1B2M2Y8AsgTpgAmY7PhCfg==,2,20239,1lkow/+J96atpaza+bjZwQ==,6825,6825,0,-1,-6825,6825,0,V-002809,fYhwLN+cpy+roem/6mSQXg==,2021-11-04,2021-11-04,2021-12-09 18:22:58,2021-12-10
geRi+pxC0L/61P7xBaJdPA==,58300,GbTAkJACGOjrVzQghLqbWg==,6825,2021-12-01,75155,SNfyN9QHUE4npoS72Hd1Bw==,506509,/8yT7c/TpIKmxe0K8MmSNA==,0,6825,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,2021-12-01,P,vItcRirL0sTcu0nfSOts+w==,3,20239,1lkow/+J96atpaza+bjZwQ==,6825,6825,0,1,6825,0,6825,V-002809,fYhwLN+cpy+roem/6mSQXg==,2021-11-04,2021-11-04,2021-12-10 11:31:18,2021-12-10
SYe36P3OgdN7+/ocUIlQJw==,20000,HtcR/HtKU0hM5hLuZcjjuA==,378.38,2021-12-10,40933,SNfyN9QHUE4npoS72Hd1Bw==,506993,/8yT7c/TpIKmxe0K8MmSNA==,0,378.38,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,2021-12-10,P,j13Eh8tAgyBPUNi2GaU8ow==,51,,SYjPoEyQZSe8RTfgN62BvA==,0,14000,14000,1,378.38,0,378.38,V-003017,Fik0+VTeuZo+2L4rH/Y5GA==,2021-12-10,2021-12-10,2021-12-10 12:05:54,2021-12-10
RfwHVmMm2yi5pzPHaHgfwA==,20000,HtcR/HtKU0hM5hLuZcjjuA==,378.38,2021-12-10,40933,SNfyN9QHUE4npoS72Hd1Bw==,506993,/8yT7c/TpIKmxe0K8MmSNA==,0,378.38,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,2021-12-10,P,j13Eh8tAgyBPUNi2GaU8ow==,63,,SYjPoEyQZSe8RTfgN62BvA==,0,14000,14000,1,378.38,0,378.38,V-003017,Fik0+VTeuZo+2L4rH/Y5GA==,2021-12-10,2021-12-10,2021-12-10 12:05:54,2021-12-10
R+r/suM2XEm3pV4OpKEUAA==,20000,HtcR/HtKU0hM5hLuZcjjuA==,378.38,2021-12-10,40933,SNfyN9QHUE4npoS72Hd1Bw==,506993,/8yT7c/TpIKmxe0K8MmSNA==,0,378.38,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,2021-12-10,P,j13Eh8tAgyBPUNi2GaU8ow==,15,,SYjPoEyQZSe8RTfgN62BvA==,0,14000,14000,1,378.38,0,378.38,V-003017,Fik0+VTeuZo+2L4rH/Y5GA==,2021-12-10,2021-12-10,2021-12-10 12:05:54,2021-12-10
8eKBstZUwA0YkRjpTduzxw==,20000,HtcR/HtKU0hM5hLuZcjjuA==,378.38,2021-12-10,40933,SNfyN9QHUE4npoS72Hd1Bw==,506993,/8yT7c/TpIKmxe0K8MmSNA==,0,378.38,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,2021-12-10,P,j13Eh8tAgyBPUNi2GaU8ow==,25,,SYjPoEyQZSe8RTfgN62BvA==,0,14000,14000,1,378.38,0,378.38,V-003017,Fik0+VTeuZo+2L4rH/Y5GA==,2021-12-10,2021-12-10,2021-12-10 12:05:54,2021-12-10
LbAdQij3k+nNd4i0+vLJLw==,20000,HtcR/HtKU0hM5hLuZcjjuA==,378.38,2021-12-10,40933,SNfyN9QHUE4npoS72Hd1Bw==,506993,/8yT7c/TpIKmxe0K8MmSNA==,0,378.38,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,2021-12-10,P,j13Eh8tAgyBPUNi2GaU8ow==,37,,SYjPoEyQZSe8RTfgN62BvA==,0,14000,14000,1,378.38,0,378.38,V-003017,Fik0+VTeuZo+2L4rH/Y5GA==,2021-12-10,2021-12-10,2021-12-10 12:05:54,2021-12-10
lztVVJ9OI+1P5nTsHkyJOg==,20000,HtcR/HtKU0hM5hLuZcjjuA==,378.38,2021-12-10,40933,SNfyN9QHUE4npoS72Hd1Bw==,506993,/8yT7c/TpIKmxe0K8MmSNA==,0,378.38,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,2021-12-10,P,j13Eh8tAgyBPUNi2GaU8ow==,49,,SYjPoEyQZSe8RTfgN62BvA==,0,14000,14000,1,378.38,0,378.38,V-003017,Fik0+VTeuZo+2L4rH/Y5GA==,2021-12-10,2021-12-10,2021-12-10 12:05:54,2021-12-10
recordno,accountno,accounttitle,amount,batch_date,batch_no,batch_title,batchkey,bookid,creditamount,debitamount,customerid,customername,description,docnumber,entry_date,entry_state,entrydescription,line_no,recordid,recordtype,totaldue,totalentered,totalpaid,tr_type,trx_amount,trx_creditamount,trx_debitamount,vendorid,vendorname,whencreated,whendue,whenmodified,whenpaid,_fivetran_deleted
YGy2nQ7YHphQv5+5AdQTyQ==,15030,A+5CoJT/HDmjwrGYGL6RUg==,2492.5,2021-12-01,74885,El+ECK1x7a25SuLqaMwODA==,506019,/8yT7c/TpIKmxe0K8MmSNA==,0,2492.5,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,06m0www4a75rDMeUgh3o0w==,2021-12-01,P,06m0www4a75rDMeUgh3o0w==,3,20760,1lkow/+J96atpaza+bjZwQ==,2492.5,2492.5,0,1,2492.5,0,2492.5,V-005638,NJa+iTk8TGFrCNzU4GaQMg==,2021-11-16,2021-11-16,2021-12-10 11:47:03,2021-12-10,FALSE
GA/oILgpDw/6ay94J1jgLA==,20000,HtcR/HtKU0hM5hLuZcjjuA==,-2492.5,2021-12-01,74885,El+ECK1x7a25SuLqaMwODA==,506019,/8yT7c/TpIKmxe0K8MmSNA==,2492.5,0,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,06m0www4a75rDMeUgh3o0w==,2021-12-01,P,1B2M2Y8AsgTpgAmY7PhCfg==,2,20760,1lkow/+J96atpaza+bjZwQ==,2492.5,2492.5,0,-1,-2492.5,2492.5,0,V-005638,NJa+iTk8TGFrCNzU4GaQMg==,2021-11-16,2021-11-16,2021-12-08 16:00:57,2021-12-10,FALSE
aA1UkdHgMiUM9APkcv3MXA==,20000,HtcR/HtKU0hM5hLuZcjjuA==,-6825,2021-12-01,75155,SNfyN9QHUE4npoS72Hd1Bw==,506509,/8yT7c/TpIKmxe0K8MmSNA==,6825,0,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,2021-12-01,P,1B2M2Y8AsgTpgAmY7PhCfg==,2,20239,1lkow/+J96atpaza+bjZwQ==,6825,6825,0,-1,-6825,6825,0,V-002809,fYhwLN+cpy+roem/6mSQXg==,2021-11-04,2021-11-04,2021-12-09 18:22:58,2021-12-10,FALSE
geRi+pxC0L/61P7xBaJdPA==,58300,GbTAkJACGOjrVzQghLqbWg==,6825,2021-12-01,75155,SNfyN9QHUE4npoS72Hd1Bw==,506509,/8yT7c/TpIKmxe0K8MmSNA==,0,6825,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,2021-12-01,P,vItcRirL0sTcu0nfSOts+w==,3,20239,1lkow/+J96atpaza+bjZwQ==,6825,6825,0,1,6825,0,6825,V-002809,fYhwLN+cpy+roem/6mSQXg==,2021-11-04,2021-11-04,2021-12-10 11:31:18,2021-12-10,FALSE
SYe36P3OgdN7+/ocUIlQJw==,20000,HtcR/HtKU0hM5hLuZcjjuA==,378.38,2021-12-10,40933,SNfyN9QHUE4npoS72Hd1Bw==,506993,/8yT7c/TpIKmxe0K8MmSNA==,0,378.38,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,2021-12-10,P,j13Eh8tAgyBPUNi2GaU8ow==,51,,SYjPoEyQZSe8RTfgN62BvA==,0,14000,14000,1,378.38,0,378.38,V-003017,Fik0+VTeuZo+2L4rH/Y5GA==,2021-12-10,2021-12-10,2021-12-10 12:05:54,2021-12-10,FALSE
RfwHVmMm2yi5pzPHaHgfwA==,20000,HtcR/HtKU0hM5hLuZcjjuA==,378.38,2021-12-10,40933,SNfyN9QHUE4npoS72Hd1Bw==,506993,/8yT7c/TpIKmxe0K8MmSNA==,0,378.38,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,2021-12-10,P,j13Eh8tAgyBPUNi2GaU8ow==,63,,SYjPoEyQZSe8RTfgN62BvA==,0,14000,14000,1,378.38,0,378.38,V-003017,Fik0+VTeuZo+2L4rH/Y5GA==,2021-12-10,2021-12-10,2021-12-10 12:05:54,2021-12-10,FALSE
R+r/suM2XEm3pV4OpKEUAA==,20000,HtcR/HtKU0hM5hLuZcjjuA==,378.38,2021-12-10,40933,SNfyN9QHUE4npoS72Hd1Bw==,506993,/8yT7c/TpIKmxe0K8MmSNA==,0,378.38,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,2021-12-10,P,j13Eh8tAgyBPUNi2GaU8ow==,15,,SYjPoEyQZSe8RTfgN62BvA==,0,14000,14000,1,378.38,0,378.38,V-003017,Fik0+VTeuZo+2L4rH/Y5GA==,2021-12-10,2021-12-10,2021-12-10 12:05:54,2021-12-10,FALSE
8eKBstZUwA0YkRjpTduzxw==,20000,HtcR/HtKU0hM5hLuZcjjuA==,378.38,2021-12-10,40933,SNfyN9QHUE4npoS72Hd1Bw==,506993,/8yT7c/TpIKmxe0K8MmSNA==,0,378.38,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,2021-12-10,P,j13Eh8tAgyBPUNi2GaU8ow==,25,,SYjPoEyQZSe8RTfgN62BvA==,0,14000,14000,1,378.38,0,378.38,V-003017,Fik0+VTeuZo+2L4rH/Y5GA==,2021-12-10,2021-12-10,2021-12-10 12:05:54,2021-12-10,FALSE
LbAdQij3k+nNd4i0+vLJLw==,20000,HtcR/HtKU0hM5hLuZcjjuA==,378.38,2021-12-10,40933,SNfyN9QHUE4npoS72Hd1Bw==,506993,/8yT7c/TpIKmxe0K8MmSNA==,0,378.38,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,2021-12-10,P,j13Eh8tAgyBPUNi2GaU8ow==,37,,SYjPoEyQZSe8RTfgN62BvA==,0,14000,14000,1,378.38,0,378.38,V-003017,Fik0+VTeuZo+2L4rH/Y5GA==,2021-12-10,2021-12-10,2021-12-10 12:05:54,2021-12-10,FALSE
lztVVJ9OI+1P5nTsHkyJOg==,20000,HtcR/HtKU0hM5hLuZcjjuA==,378.38,2021-12-10,40933,SNfyN9QHUE4npoS72Hd1Bw==,506993,/8yT7c/TpIKmxe0K8MmSNA==,0,378.38,,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,1B2M2Y8AsgTpgAmY7PhCfg==,2021-12-10,P,j13Eh8tAgyBPUNi2GaU8ow==,49,,SYjPoEyQZSe8RTfgN62BvA==,0,14000,14000,1,378.38,0,378.38,V-003017,Fik0+VTeuZo+2L4rH/Y5GA==,2021-12-10,2021-12-10,2021-12-10 12:05:54,2021-12-10,TRUE
3 changes: 2 additions & 1 deletion macros/get_gl_detail_columns.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@
{"name": "whencreated", "datatype": "date"},
{"name": "whendue", "datatype": "date"},
{"name": "whenmodified", "datatype": dbt.type_timestamp()},
{"name": "whenpaid", "datatype": "date"}
{"name": "whenpaid", "datatype": "date"},
{"name": "_fivetran_deleted", "datatype": "boolean"}
] %}
{{ return(columns) }}
{% endmacro %}
2 changes: 2 additions & 0 deletions models/stg_sage_intacct.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -413,6 +413,8 @@ models:
description: When transaction is modified
- name: paid_at
description: When transaction was paid
- name: _fivetran_deleted
description: The timestamp of when the data was deleted by Fivetran

- name: stg_sage_intacct__gl_batch
description: Records batch updates of records originating from the gl_detail table
Expand Down
3 changes: 2 additions & 1 deletion models/stg_sage_intacct__gl_detail.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,8 @@ final as (
whencreated as created_at,
whendue as due_at,
whenmodified as modified_at,
whenpaid as paid_at
whenpaid as paid_at,
_fivetran_deleted


--The below script allows for pass through columns.
Expand Down