-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
codec: add simple protocol #1139
base: master
Are you sure you want to change the base?
Conversation
Skipping CI for Draft Pull Request. |
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
/retest |
Have you support |
This reverts commit 3f455a7.
@wk989898: The following test failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR introduces and integrates a simple protocol for the codec component, updating event type usage and variable references to align with the new protocol.
- Refactored marshaller and encoder implementations to use commonEvent types instead of model types.
- Updated table ID and schema version handling across components.
- Removed the legacy simple protocol decoder and adapted bootstrap and encoder_group logic accordingly.
Reviewed Changes
Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.
Show a summary per file
File | Description |
---|---|
pkg/sink/codec/simple/mock/marshaller.go | New autogenerated mock implementation for the marshaller interface. |
pkg/sink/codec/simple/marshaller.go | Updates to interface signatures and config usage for simple protocol. |
pkg/sink/codec/simple/encoder.go | Replaced commented legacy code with active implementation using common. |
pkg/sink/codec/simple/decoder.go | Removed the decoder, reflecting protocol deprecation. |
pkg/sink/codec/simple/avro.go | Adjusted Avro schema mappings and DML message generation. |
pkg/sink/codec/encoder_group.go | Updated bootstrap event handling using the new RowEvent type. |
pkg/sink/codec/encoder_builder.go | Modified protocol selection to instantiate simple protocol encoder. |
pkg/sink/codec/common/utils.go | Added test utilities for large event generation. |
pkg/sink/codec/bootstraper.go | Updated bootstrap logic to use the new table ID property from RowEvent. |
pkg/common/event/row_change.go | Added PhysicalTableID and its accessor to support simple protocol changes. |
pkg/common/event/ddl_event.go | Added an IsBootstrap flag to the DDL event for protocol distinction. |
downstreamadapter/worker/mq_dml_worker.go | Propagated PhysicalTableID from events to downstream adapter processing. |
Files not reviewed (1)
- Makefile: Language not supported
Comments suppressed due to low confidence (3)
pkg/common/event/row_change.go:88
- [nitpick] Consider adding a comment to explain the purpose and usage of the 'PhysicalTableID' field in the RowEvent struct.
PhysicalTableID int64
pkg/sink/codec/simple/avro.go:194
- Directly accessing ddl.MultipleTableInfos[1] could lead to an index out-of-range error; consider adding an explicit check to ensure the index exists or use a safer approach to retrieve the previous table info.
preTableInfo := ddl.MultipleTableInfos[1] // previous table info, TODO: need check it
pkg/sink/codec/bootstraper.go:232
- Ensure that 'row.TableInfo' is non-nil before accessing 'row.TableInfo.TableName.TableID' to prevent potential runtime panics.
id: row.TableInfo.TableName.TableID,
// return errors.Trace(err) | ||
// } | ||
// } | ||
if g.bootstrapWorker != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Validate that the 'events' slice is non-empty before accessing events[0] in bootstrapWorker.addEvent() to avoid potential out-of-range panics.
if g.bootstrapWorker != nil { | |
if g.bootstrapWorker != nil { | |
if len(events) == 0 { | |
return errors.New("no events to add") | |
} |
Copilot is powered by AI, so mistakes are possible. Review output carefully before use.
What problem does this PR solve?
Issue Number: ref #1189
What is changed and how it works?
Check List
Tests
Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note