Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add IcebergCatalog rest support #67

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

liko9
Copy link

@liko9 liko9 commented Jun 24, 2024

Added ability to lake loader to support writing to Iceberg tables managed by a rest-based catalog. Tested successfully with the Tabular implementation located at https://github.com/tabular-io/iceberg-rest-image and AWS S3. Should work with GCP and Azure too. Helps advance #22

@snowplowcla
Copy link

Thanks for your pull request. Is this your first contribution to a Snowplow open source project? Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://docs.snowplowanalytics.com/docs/contributing/contributor-license-agreement/ to learn more and sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.

@liko9
Copy link
Author

liko9 commented Jun 24, 2024

Thanks for your pull request. Is this your first contribution to a Snowplow open source project? Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://docs.snowplowanalytics.com/docs/contributing/contributor-license-agreement/ to learn more and sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.

I signed. I hope all lawyers are happy. :)

@snowplowcla
Copy link

Confirmed! @liko9 has signed the Contributor License Agreement. Thanks so much.

@miike miike requested a review from istreeter June 26, 2024 01:25
@stanch
Copy link

stanch commented Jul 8, 2024

Hi @liko9, thanks for this! To fully support Rest in our product, we will need to make more changes (including private repos) — things like creating new artifacts, updating Terraform, surfacing this in our UI, and so on. We will come back to this PR once everything is in place.

Also note that currently we only support Iceberg on AWS, where Glue catalog is probably the most popular one. On GCP/Azure there previously was not a good choice, but Rest is changing that :) Still, we have a few more things to wrap up before we start working on GCP/Azure support (or non-Glue AWS support).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants