Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support for bucketed Acid tables #98

Open
srinikvv opened this issue Oct 8, 2020 · 0 comments
Open

Support for bucketed Acid tables #98

srinikvv opened this issue Oct 8, 2020 · 0 comments

Comments

@srinikvv
Copy link

srinikvv commented Oct 8, 2020 •
edited
Loading

We are using Merge on Hive Acid tables (non-bucketed) to maintain SCD Type-1 data with incremental updates from our sources. Over a period of time, regular merge statements on this table resulted in lot small files (even after compaction). This has a direct impact on dependent ETLs and queries. However this issue could be avoided with support for bucketed tables since the data would always be split into pre-defined # buckets post compaction.

@amoghmargoor above limitation is causing a lot of instability in the hive warehouse. Kindly check the possibility of supporting bucketed tables in MERGE.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@srinikvv