Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clp integration mysql metadata #1

Open
wants to merge 85 commits into
base: master
Choose a base branch
from

Conversation

wraymo
Copy link

@wraymo wraymo commented Feb 19, 2025

Description

Motivation and Context

Impact

Test Plan

Contributor checklist

  • Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

General Changes
* ... :pr:`12345`
* ... :pr:`12345`

Hive Connector Changes
* ... :pr:`12345`
* ... :pr:`12345`

If release note is NOT required, use:

== NO RELEASE NOTE ==

Comment on lines +334 to +348
catch (SQLException e) {
log.error(e, "Failed to connect to metadata database");
return ImmutableList.of();
}
finally {
// Closing the connection
try {
if (connection != null) {
connection.close();
}
}
catch (SQLException ex) {
log.warn(ex, "Failed to close metadata database connection");
}
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems such try-catch is heavily used when doing SQL operations. Can we wrap it into a function or class (say MetadataQueryRunner), for example, which takes a function call as parameter.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This finally code block was removed

private String getVariableName(VariableReferenceExpression variable)
{
String variableName = ((ClpColumnHandle) assignments.get(variable)).getColumnName();
if (variableName.endsWith("_bigint") || variableName.endsWith("_double") ||
Copy link

@anlowee anlowee Feb 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we move these constant strings "_bigint" etc. into ClpColumnHandle. Actually, I figure we can move this function too ClpColumnHandle, make it a member method of it

return variableName;
}

private ClpExpression handleNot(CallExpression node)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we add a TODO comment in each of these handle* functions? Say we will support more complicated logic expression in the future.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you show an example of more complicated logic? I don't think KQL supports more

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for example more logic operators can work in a row (A AND B AND C OR D ...). If KQL doesn't not support that shall we mention that in the TODO, because this expects to be one of the basic feature of regular SQL

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it supports such logic. It's basically AND and OR, no new operators.

@wraymo
Copy link
Author

wraymo commented Mar 17, 2025 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants