[Feature] Support a time-aware compaction strategy for historical partitions in PK Table

[xiangyuf]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

For unordered updates in a partitioned primary key table, we can reduce the compaction frequency for historical partitions to reduce the overall resource usage.

Especially for deletion vector table and lookup changelog-producer table, which will submit a new compaction task for each new created sorted-run in L0 level. In such cases, users can actually accept worse data freshness for the late arrived data in the historical partitions for better overall resource usage.

Solution

No response

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!