Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support MySQL vector type #1004

Merged
merged 2 commits into from
Mar 6, 2025
Merged

Support MySQL vector type #1004

merged 2 commits into from
Mar 6, 2025

Conversation

serprex
Copy link
Contributor

@serprex serprex commented Feb 28, 2025

No description provided.

@serprex serprex marked this pull request as ready for review March 4, 2025 19:33
@serprex
Copy link
Contributor Author

serprex commented Mar 4, 2025

I've verified this works on peerdb, would be good to get tests, but that's blocked on using a more recent version of mysql in unit tests. Would like to look into widening CI separately

@dveeden
Copy link
Collaborator

dveeden commented Mar 5, 2025

With MySQL 9.1.0:

reset binary logs and gtids;
create table t1 (id int primary key auto_increment, v vector);
insert into t1(v) values (STRING_TO_VECTOR("[1.05, -17.8, 32]"));

Without this PR:

Get event error: Header &replication.EventHeader{Timestamp:0x67c85160, EventType:0x1e, ServerID:0x1, EventSize:0x38, LogPos:0x2e4, Flags:0x0}, Data "\x81\x02\x00\x00\x00\x00\x01\x00\x02\x00\x02\xff\x00\x01\x00\x00\x00\f\x00\x00\x00ff\x86?ff\x8e\xc1\x00\x00\x00B", Err: unsupport type 242 in binlog and don't know how to handle
github.com/pingcap/errors.AddStack
	/home/dvaneeden/go/pkg/mod/github.com/pingcap/[email protected]/errors.go:174
github.com/pingcap/errors.Trace
	/home/dvaneeden/go/pkg/mod/github.com/pingcap/[email protected]/juju_adaptor.go:15
github.com/go-mysql-org/go-mysql/replication.(*BinlogSyncer).parseEvent
	/home/dvaneeden/dev/go-mysql/replication/binlogsyncer.go:843
github.com/go-mysql-org/go-mysql/replication.(*BinlogSyncer).onStream
	/home/dvaneeden/dev/go-mysql/replication/binlogsyncer.go:796
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1700

With this PR:

=== TableMapEvent ===
Date: 2025-03-05 14:28:00
Log position: 684
Event size: 58
TableID: 641
TableID size: 6
Flags: 1
Schema: test
Table: t1
Column count: 2
Column type: 
00000000  03 f2                                             |..|
NULL bitmap: 
00000000  02                                                |.|
Signedness bitmap: 
00000000  00                                                |.|
Default charset: [63]
Column charset: []
Set str value: []
Enum str value: []
Column name: []
Geometry type: []
Primary key: []
Primary key prefix: []
Enum/set default charset: []
Enum/set column charset: []
Invisible Column bitmap: 
UnsignedMap: map[int]bool{0:false}
CollationMap: map[int]uint64{}
EnumSetCollationMap: map[int]uint64(nil)
EnumStrValueMap: map[int][]string(nil)
SetStrValueMap: map[int][]string(nil)
GeometryTypeMap: map[int]uint64(nil)
VisibilityMap: map[int]bool(nil)
Columns: 
  <n/a>  type=3    unsigned=no   null=no 
  <n/a>  type=242  null=yes

=== WriteRowsEventV2 ===
Date: 2025-03-05 14:28:00
Log position: 740
Event size: 56
TableID: 641
Flags: 1
Column count: 2
NDB data: 
Values:
--
0:1
1:"ff\x86?ff\x8e\xc1\x00\x00\x00B"

=== XIDEvent ===
Date: 2025-03-05 14:28:00
Log position: 771
Event size: 31
XID: 5772

@lance6716
Copy link
Collaborator

lance6716 commented Mar 6, 2025

I've verified this works on peerdb, would be good to get tests, but that's blocked on using a more recent version of mysql in unit tests. Would like to look into widening CI separately

we can create an issue in this repo tp enhance CI coverage on different MySQL distro or versions. I agree that we can fix it later.

@lance6716 lance6716 merged commit 042dd6c into go-mysql-org:master Mar 6, 2025
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants