Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flink write-parallelism causes distribution-mode: RANGE to not work properly #11543

Open
1 of 3 tasks
huyuanfeng2018 opened this issue Nov 14, 2024 · 0 comments
Open
1 of 3 tasks
Labels
bug Something isn't working

Comments

@huyuanfeng2018
Copy link
Contributor

Apache Iceberg version

1.7.0 (latest release)

Query engine

Flink

Please describe the bug 🐞

When we set write-parallelism, if the parallelism of writing is different from the parallelism of tasks, iceberg write operator will be disconnected. In this case, distribution-mode: RANGE will have no effect.

image

We should throw this exception when the task jobgraph is generated

cc @stevenzwu @pvary

Willingness to contribute

  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time
@huyuanfeng2018 huyuanfeng2018 added the bug Something isn't working label Nov 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant