exclude files from being treated as folders in table_definitions_folder (#209)

charlielewisme · web-flow · commit 1df5b6c11ad1 · 2024-09-02T11:51:23.000+01:00
diff --git a/airflow/dags/polygon_parse_dag.py b/airflow/dags/polygon_parse_dag.py
@@ -21,6 +21,8 @@
 )
 
 for folder in glob(table_definitions_folder):
+    if not os.path.isdir(folder):
+        continue
     dataset = folder.split('/')[-1]
 
     dag_id = f'polygon_parse_{dataset}_dag'

Original file line number	Diff line number	Diff line change
`@@ -21,6 +21,8 @@`
`21`	`21`	`)`
`22`	`22`
`23`	`23`	`for folder in glob(table_definitions_folder):`
	`24`	`+ if not os.path.isdir(folder):`
	`25`	`+ continue`
`24`	`26`	`dataset = folder.split('/')[-1]`
`25`	`27`
`26`	`28`	`dag_id = f'polygon_parse_{dataset}_dag'`