-
Notifications
You must be signed in to change notification settings - Fork 299
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[#4952] feat(hudi-catalog): add implementation of HMSBackend for Hudi catalog #4942
base: main
Are you sure you want to change the base?
Conversation
.put( | ||
LOCATION, | ||
PropertyEntry.stringOptionalPropertyEntry( | ||
LOCATION, "The directory for Hudi dataset storage", false, null, false)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we please follow the style to add comment like /* required */
to the input variables.
.put( | ||
LOCATION, | ||
stringImmutablePropertyEntry( | ||
LOCATION, "The location for Hudi table", false, null, false, false)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also here.
c -> | ||
c.getTables(schemaIdent.name(), "*").stream() | ||
.map(table -> NameIdentifier.of(namespace, table)) | ||
.toArray(NameIdentifier[]::new)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will here list all the tables, not just Hudi table, should we filter out non-hudi table?
partitioning = HiveTableConverter.getPartitioning(hmsTable); | ||
sortOrders = HiveTableConverter.getSortOrders(hmsTable); | ||
distribution = HiveTableConverter.getDistribution(hmsTable); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does Hudi store such information in HMS, is it compatible with Hive table? As I know, for Iceberg, we need some Iceberg APIs to get partitioning, sortOrders, because Iceberg will store such information in it's metadata file, not in HMS, I guess Hudi is similar, can you please confirm this?
What changes were proposed in this pull request?
support read operations for Hudi catalog HMS backend
Why are the changes needed?
Fix: #4952
Does this PR introduce any user-facing change?
no
How was this patch tested?
UTs added