mysql #2395
mysql #2395
Conversation
serprex
force-pushed
the
mysql
branch
25 times, most recently
from
933bd2c to
85394e3
Compare
serprex
force-pushed
the
mysql
branch
13 times, most recently
from
4fea228 to
319ebd3
Compare
split out from #2395, where this allows tests to run against both PG & MySQL
serprex
force-pushed
the
mysql
branch
2 times, most recently
from
f40244b to
54d141b
Compare
flow/connectors/mysql/cdc.go
Outdated
| qkind = qvalue.QValueKindInt64 | ||
| case mysql.MYSQL_TYPE_SET: | ||
| qkind = qvalue.QValueKindInt64 | ||
| case mysql.MYSQL_TYPE_TINY_BLOB: |
There was a problem hiding this comment.
nit: unless you want different types down the line, can coalesce a lot of the case statements together
| - 3306:3306 | ||
| env: | ||
| MYSQL_ROOT_PASSWORD: cipass | ||
| #mariadb: |
There was a problem hiding this comment.
TODO. I could not get CI to play nice with both services listening on 3306 even when trying to change port mapping
| @@ -0,0 +1,3 @@ | |||
| ALTER TABLE metadata_last_sync_state ADD COLUMN IF NOT EXISTS last_text text NOT NULL DEFAULT ''; | |||
There was a problem hiding this comment.
wondering if you can store Pg and My states in a single column and run a migration in this file, shouldn't be too many rows
There was a problem hiding this comment.
No. PG has GREATEST logic around the integer value, whereas mysql does not
| @@ -142,6 +142,7 @@ message MySqlConfig { | |||
| repeated string setup = 6; | |||
| uint32 compression = 7; | |||
| bool disable_tls = 8; | |||
| string flavor = 9; | |||
There was a problem hiding this comment.
could be an enum at the proto level?
| return nil, nil | ||
| } | ||
|
|
||
| func (c *MySqlConnector) AddTablesToPublication(ctx context.Context, req *protos.AddTablesToPublicationInput) error { |
There was a problem hiding this comment.
is there an analogue to this operation in MySQL at all? if not should be moved out of interface
same for below
There was a problem hiding this comment.
There's not, consumer is expected to filter. Moving stuff out of interface can be done in followup, there's plenty of places where we go with no-op implementations so that cleanup could be done more generally
| } | ||
|
|
||
| func (c *MySqlConnector) GetMasterPos(ctx context.Context) (mysql.Position, error) { | ||
| showBinlogStatus := "SHOW BINARY LOG STATUS" |
There was a problem hiding this comment.
I believe this is flavour dependent as well: https://mariadb.com/kb/en/show-binlog-status/
There was a problem hiding this comment.
Need to have version checks be per-flavor too
|
|
||
| rr, err := c.Execute(ctx, showBinlogStatus) | ||
| if err != nil { | ||
| return mysql.Position{}, fmt.Errorf("failed to SHOW BINARY LOG STATUS: %w", err) |
There was a problem hiding this comment.
inject actual command run here
| case nil: | ||
| return qvalue.QValueNull(qkind), nil | ||
| case uint64: | ||
| // TODO unsigned integers |
There was a problem hiding this comment.
TODO or in a followup?
There was a problem hiding this comment.
followup, most TODOs here at this point are looking to get resolved after merge
| @@ -0,0 +1,117 @@ | |||
| package connmysql | |||
There was a problem hiding this comment.
I'm not sure why this is needed
There was a problem hiding this comment.
binsyncer in go-mysql uses https://github.com/siddontang/go-log will add comment linking to this
for pg we currently use destination instead of catalog. mysql->pg will use catalog therefore metadata tables on destination pg can ignore this change, avoiding issues upgrading existing pgpg setups also update connector interfaces a bit split out from #2395
serprex
force-pushed
the
mysql
branch
2 times, most recently
from
e67454b to
83d88a4
Compare
split from #2395 this will reduce the only changed lines to specialized lastOffset in flowable_core.go
No description provided.