I think we should limit total buckets?

This is also my initial purpose, but I found that the bucket generation logic in BucketAssigner is a instance behavior and could be parallelized, every assigner handles their own buckets. On the other hand, This feature is equivalent to the total bucket limit by default (1 parallellism).
Furthuremore, the universal bucket limit may cause some assigners to work less, resulting in CPU and data skew. WDYT?

I still think introducing a max bucket is better.
The option should be very easy to understand.

+1
Add option should be easy to user understand.

Reasonable enough, I have found a way to implement total max bucket feature, thanks for reminding that :)

    public static int[] distributeBucketsToAssigners(int maxBuckets, int assigners) {
        int[] result = new int[assigners];
        int avg = maxBuckets / assigners;
        int remainder = maxBuckets % assigners;
        for (int i = 0; i < assigners; i++) {
            result[i] = avg;
            if (remainder > 0) {
                result[i]++;
                remainder--;
            }
        }
        return result;
    }

Add a util function of PartitionIndex#cacheBucketAndGet, then here use?

+1
Add option should be easy to user understand.