[Bug] New Partiions created will cause Flink job failover when partition discovery is enabled while scanning

[luoyuxia]

Search before asking

• I searched in the issues and found nothing similar.

Fluss version

0.5.0

Minimal reproduce step

Create a partitioned table and a Flink job to subscribe it.. After new partitions is created, it may well throw

Caused by: com.alibaba.fluss.exception.FlussRuntimeException: Leader not found for table bucket: TableBucket{tableId=0, partitionId=60, bucket=16}
	at com.alibaba.fluss.client.metadata.MetadataUpdater.leaderFor(MetadataUpdater.java:122)
	at com.alibaba.fluss.client.admin.FlussAdmin.prepareListOffsetsRequests(FlussAdmin.java:295)
	at com.alibaba.fluss.client.admin.FlussAdmin.listOffsets(FlussAdmin.java:253)
	at com.alibaba.fluss.connector.flink.source.enumerator.initializer.BucketOffsetsRetrieverImpl.listOffsets(BucketOffsetsRetrieverImpl.java:72)
	at com.alibaba.fluss.connector.flink.source.enumerator.initializer.BucketOffsetsRetrieverImpl.latestOffsets(BucketOffsetsRetrieverImpl.java:49)
	at com.alibaba.fluss.connector.flink.source.enumerator.initializer.LatestOffsetsInitializer.getBucketOffsets(LatestOffsetsInitializer.java:38)
	at com.alibaba.fluss.connector.flink.source.enumerator.FlinkSourceEnumerator.getLogSplit(FlinkSourceEnumerator.java:449)
	at com.alibaba.fluss.connector.flink.source.enumerator.FlinkSourceEnumerator.initLogTablePartitionSplits(FlinkSourceEnumerator.java:356)
	at com.alibaba.fluss.connector.flink.source.enumerator.FlinkSourceEnumerator.initPartitionedSplits(FlinkSourceEnumerator.java:348)
	at com.alibaba.fluss.connector.flink.source.enumerator.FlinkSourceEnumerator.lambda$checkPartitionChanges$2(FlinkSourceEnumerator.java:296)
	at org.apache.flink.runtime.source.coordinator.ExecutorNotifier.lambda$notifyReadyAsync$2(ExecutorNotifier.java:80)

It'll cause job failover

What doesn't meet your expectations?

It's in the expectation that throws an exception since the leader may well still not be elected although partition has been created...Although it'll running normally after failover, we'd better to handle this cause to avoid failover again and again in every partition created.

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!