Hello, I am conducting tests with Z-order optimization and the min_commit_interval. It seems that while optimize respects the min_commit_interval, the Z-order operation does not use this setting as expected.

[ion-elgreco]

Hello, I am conducting tests with Z-order optimization and the min_commit_interval. It seems that while optimize respects the min_commit_interval, the Z-order operation does not use this setting as expected.

Each Z-order operation is creating a new entry in the Delta log, even when min_commit_interval is set to a higher timedelta value. This is not happening with optimize.compact.

My expectation was that Z-order would batch commits based on min_commit_interval, similar to how optimize works, but this does not appear to be the case.

Environment:

Delta Lake version: 0.21.0

Originally posted by @carlos6moya in #1645 (comment)