BigQueryToGCSOperator for dataset instead of single table?

[Rittycheriah]

Description

Hi,
Recently, there was a community dev discussion about how to snapshot tables for longer than 7 days in BigQuery. One solution that worked well for me in a previous context was leveraging the BigQueryToGCSOperator logic, but instead of employing it per table, the top level DAG submitted a dataset name and then I could loop over all the available tables and export to GCS as a backup. Would it be helpful to others for me to build an open source version of this logic? Just a thought! Thank y'all.

Use case/motivation

I'd like to contribute to the Airflow project by making a per BigQuery dataset to GCS folder operator a reality. If folks are interested in snapshotting tables for backups, this could pose an easier way to do it than directly in BigQuery itself.

Related issues

None than I'm aware of!

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[potiuk]

Feel free.

[Rittycheriah]

Started in on implementation here, but working on getting the dev env up to test. Lmk if this seems like the proper direction to you. No rush! #37695