docs: add standalone partition pages in user guide #3192
Conversation
|
ACTION NEEDED delta-rs follows the Conventional Commits specification for release automation. The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification. |
braaannigan
force-pushed
the
github-pages
branch
from
c39c572 to
d7e13eb
Compare
braaannigan
changed the title
| ( | ||
| dt.merge( | ||
| source=source_data, | ||
| predicate="target.country = source.country AND target.num = source.num", |
There was a problem hiding this comment.
When you merge on partitions and you are certain the source data only holds one partition or multiple, then you can should an explicit partition predicate. Especially when we have streaming mode enabled by default
There was a problem hiding this comment.
Hi @ion-elgreco, can I confirm I understand what you are suggesting here: that the code is fine but we clarify in the text that having partitions in the predicate is very important for performance if only a subset of partitions are likely to be matched?
There was a problem hiding this comment.
@ion-elgreco I've re-worked this entire section to emphasise the points as I understood them
There was a problem hiding this comment.
I meant, s.id = t.id AND t.id in (1,2,3)
There was a problem hiding this comment.
@ion-elgreco Ah, I didn't know that. We're all learning something!
I've re-worked that section accordingly
Signed-off-by: Liam Brannigan <[email protected]>
Signed-off-by: Liam Brannigan <[email protected]>
Signed-off-by: Liam Brannigan <[email protected]>
Signed-off-by: Liam Brannigan <[email protected]>
Signed-off-by: Martin Andersson <[email protected]> Signed-off-by: Liam Brannigan <[email protected]>
braaannigan
force-pushed
the
github-pages
branch
from
7e45ca6 to
8c45fd4
Compare
braaannigan
requested review from
wjones127,
fvaleye and
roeap
as code owners
Description
This page is a standalone user guide page that focuses on working with partitioned datasets.
Related Issue(s)
closes #3191