Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reading Delta Tables in Fabric #102

Open
bill-ash opened this issue Oct 1, 2024 · 1 comment
Open

Reading Delta Tables in Fabric #102

bill-ash opened this issue Oct 1, 2024 · 1 comment

Comments

@bill-ash
Copy link

bill-ash commented Oct 1, 2024

Getting the following error:

IOException: IO Error: Parquet file is likely corrupted, cannot have dictionary offsets without seeing a non-empty dictionary first.

when reading a delta table in Fabric. No issues when reading the same table using spark.

When I choose a subset of columns the query will complete successfully ie:

# throws an error
duckdb.sql("select * from delta_scan('/lakehouse/default/Tables/<schema>/<table_name>'")

# completes without error 
duckdb.sql("select col1, col2, from delta_scan('/lakehouse/default/Tables/<schema>/<table_name>'")

Will try to nail down the exact columns and their types that are causing issues but maybe someone else recognizes the error. Thanks!

@samansmink
Copy link
Collaborator

This should be fixed through duckdb/duckdb#14268. Would you mind trying again on duckdb v1.1.3?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants