Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat binlog apply optimization #1378

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

shaohk
Copy link
Contributor

@shaohk shaohk commented Feb 5, 2024

Description

  1. Support for ignoring binlog events that exceed chunk boundary values.
    If the value corresponding to the unique key of the binlog exceeds the maximum value of chunk iteration and is less than the maximum boundary value of the copy, it is ignored.

  2. Support for binlog merge processing.
    When the columns of the unique key selected by chunk are int or float, the binlog is processed by map merging, and all delete operations are merged into one delete sql.All insert and update operations are merged into one replace sql. Then execute these sql in db transaction.

In case this PR introduced Go code changes:

  • contributed code is using same conventions as original code
  • script/cibuild returns with no formatting errors, build errors or unit test errors.

shaohoukun added 2 commits February 5, 2024 18:21
1. Support for ignoring binlog events that exceed chunk boundary values. 2. Support for binlog merge
processing.
@shaohk shaohk changed the title (feat): Feat binlog apply optimization Feat binlog apply optimization Feb 5, 2024
shaohoukun added 3 commits February 6, 2024 14:39
When there is only one column in a unique index, merging of DML binlog events is permitted.
…tionRangeMaxValues is nil

Add handling for the case when MigrationIterationRangeMaxValues is nil
@shaohk shaohk changed the title Feat binlog apply optimization feat binlog apply optimization Mar 20, 2024
@ramshad-sha ramshad-sha mentioned this pull request Mar 25, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants