Add Iceberg to AlloyDB Blueprint

[tarun-google]

Adding Iceberg to AlloyDB blueprint

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

[claudevdm]

https://source.corp.google.com/piper///depot/google3/third_party/py/apache_beam/ml/rag/ingestion/alloydb.py;l=73-124?q=alloydb.py&sq=package:piper%20file:%2F%2Fdepot%2Fgoogle3%20-file:google3%2Fexperimental

This shows how to use the AlloyDB language connector, specifically see how the jdbc_url is formatted with instance name etc.

It also requires expansion service to have com.google.cloud:alloydb-jdbc-connector

https://github.com/GoogleCloudPlatform/alloydb-java-connector/blob/main/docs/jdbc.md for official docs

[tarun-google]

schema-io-expansion-service already contain required alloydb-jdbc-connector and postgres jars

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 56.80%. Comparing base (35969b3) to head (b1d88ea).
⚠️ Report is 5 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##             master   #36036   +/-   ##
=========================================
  Coverage     56.79%   56.80%           
  Complexity     3385     3385           
=========================================
  Files          1220     1220           
  Lines        185122   185122           
  Branches       3508     3508           
=========================================
+ Hits         105148   105154    +6     
+ Misses        76649    76643    -6     
  Partials       3325     3325           

Flag	Coverage Δ
python	81.01% <ø> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[tarun-google]

Run Python_Transforms PreCommit 3.10

[github-actions]

Assigning reviewers:

R: @shunping for label python.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[tarun-google]

R: @claudevdm @derrickaw @chamikaramj

[github-actions]

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

[liferoad]

/gemini review

[gemini-code-assist]

Code Review

This pull request adds a new Beam YAML blueprint demonstrating a pipeline that reads from an Apache Iceberg table and writes to an AlloyDB database. The changes include the YAML definition for the pipeline and updates to the Python test suite to include this new example. The overall implementation is good. I have one suggestion regarding the use of placeholder credentials in the example YAML file to improve security best practices.

[tarun-google]

Run Python_Transforms PreCommit 3.12

[tarun-google]

Run Python_Runners PreCommit 3.10

[tarun-google]

Run Python_ML PreCommit 3.11

[tarun-google]

Run Python_Transforms PreCommit 3.12