-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support MERGE on cloned table in Delta Lake #24756
Conversation
37ff93a
to
60e9e73
Compare
60e9e73
to
6432fab
Compare
9bc68fc
to
0cb664f
Compare
...sts/src/main/java/io/trino/tests/product/deltalake/TestDeltaLakeCloneTableCompatibility.java
Show resolved
Hide resolved
...sts/src/main/java/io/trino/tests/product/deltalake/TestDeltaLakeCloneTableCompatibility.java
Show resolved
Hide resolved
...sts/src/main/java/io/trino/tests/product/deltalake/TestDeltaLakeCloneTableCompatibility.java
Show resolved
Hide resolved
plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMergeSink.java
Show resolved
Hide resolved
...sts/src/main/java/io/trino/tests/product/deltalake/TestDeltaLakeCloneTableCompatibility.java
Show resolved
Hide resolved
plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMergeSink.java
Outdated
Show resolved
Hide resolved
plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMetadata.java
Outdated
Show resolved
Hide resolved
7ef5f5d
to
be52ef7
Compare
be52ef7
to
1b33203
Compare
plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMetadata.java
Outdated
Show resolved
Hide resolved
plugin/trino-delta-lake/src/test/java/io/trino/plugin/deltalake/TestDeltaLakeBasic.java
Show resolved
Hide resolved
plugin/trino-delta-lake/src/test/resources/deltalake/clone_merge/clone_merge_cloned/README.md
Outdated
Show resolved
Hide resolved
b583ce9
to
e8d8d33
Compare
e8d8d33
to
c76bcf1
Compare
plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMergeSink.java
Show resolved
Hide resolved
c76bcf1
to
5325f92
Compare
...a-lake/src/test/resources/deltalake/clone_merge/clone_merge_deletion_vector_source/README.md
Show resolved
Hide resolved
5325f92
to
93cbd29
Compare
plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMergeSink.java
Show resolved
Hide resolved
plugin/trino-delta-lake/src/test/java/io/trino/plugin/deltalake/TestDeltaLakeBasic.java
Show resolved
Hide resolved
plugin/trino-delta-lake/src/test/java/io/trino/plugin/deltalake/TestDeltaLakeBasic.java
Outdated
Show resolved
Hide resolved
93cbd29
to
2aef993
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMetadata.java
Outdated
Show resolved
Hide resolved
2577e5d
to
0b2cb43
Compare
/test-with-secrets sha=0b2cb43ae47dea47a9f7fb86d745453699188cb3 |
The CI workflow run with tests that require additional secrets has been started: https://github.com/trinodb/trino/actions/runs/13281480006 |
plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMetadata.java
Outdated
Show resolved
Hide resolved
plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMetadata.java
Outdated
Show resolved
Hide resolved
plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMetadata.java
Outdated
Show resolved
Hide resolved
plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMetadata.java
Show resolved
Hide resolved
0b2cb43
to
3a91f70
Compare
@chenjian2664 Please throw exceptions for unclear situations. This is initial merge support for cloned tables. We should start as strict as possible. |
463ab8e
to
0f5aa9c
Compare
0f5aa9c
to
f16e93c
Compare
(Rebased on master with no changes) |
Description
Fix problem that fail update on cloned table, reproduce steps:
testing/bin/ptl env up --environment singlenode-delta-lake-oss
In Trino:
create schema delta.tiny with (location='s3://test-bucket/tiny/');
In Spark-sql:
CREATE TABLE tiny.t1 (id int, v string, part date) USING DELTA PARTITIONED BY (part);
In Trino:
insert into delta.tiny.t1 values (1, 'A', TIMESTAMP '2024-01-01'), (2, 'B', TIMESTAMP '2024-01-01'), (3, 'C', TIMESTAMP '2024-02-02'), (4, 'D', TIMESTAMP '2024-02-02');
In Spark-sql:
CREATE TABLE tiny.t1clone SHALLOW CLONE tiny.t1;
In Trino:
update delta.tiny.t1clone set v = 'update1' where id in (1,3);
It fails with:Additional context and related issues
Release notes
( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text: