Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot read transaction log of partitioned table with column mapping #2914

Open
rohanjain101 opened this issue Sep 27, 2024 · 0 comments
Open
Labels
binding/python Issues for the Python package binding/rust Issues for the Rust crate bug Something isn't working

Comments

@rohanjain101
Copy link

rohanjain101 commented Sep 27, 2024

Environment

Delta-rs version: 0.20

Binding: python

Environment:

  • Cloud provider:
  • OS: windows
  • Other:

Bug

What happened:

When doing

DeltaTable("table-partitioned-column-mapping"), an error is raised like Partition column A-35fc4c98ba33431eb2c67a0b8831fbb3 not found in schema. This happens because partitionValues in commit written using physical name, for example: "partitionValues":{"A-35fc4c98ba33431eb2c67a0b8831fbb3":"1360466709"}
What you expected to happen:

I expected to be able to read the transaction log. When columnMapping enabled, partition columns should be mapped using logical instead of physical name.

How to reproduce it:

Create table with partition columns and columnMapping by name enabled, write to it, then try to create DeltaTable referencing it

More details:

@rohanjain101 rohanjain101 added the bug Something isn't working label Sep 27, 2024
@rtyler rtyler added binding/rust Issues for the Rust crate binding/python Issues for the Python package labels Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
binding/python Issues for the Python package binding/rust Issues for the Rust crate bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants