Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] AttributeError: 'SASLClient' object has no attribute 'start' #475

Open
2 tasks done
ntnhaatj opened this issue Aug 31, 2024 · 4 comments · May be fixed by dbt-labs/dbt-spark#1095
Open
2 tasks done

[Bug] AttributeError: 'SASLClient' object has no attribute 'start' #475

ntnhaatj opened this issue Aug 31, 2024 · 4 comments · May be fixed by dbt-labs/dbt-spark#1095
Labels
pkg:dbt-spark Issue affects dbt-spark triage:product In Product's queue type:bug Something isn't working as documented

Comments

@ntnhaatj
Copy link

Is this a new bug in dbt-spark?

  • I believe this is a new bug in dbt-spark
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

using thrift with enabled ssl, skip verifying by default
https://github.com/dbt-labs/dbt-spark/blob/212442322b509eb34deee2ac000c85c1e4daba73/dbt/adapters/spark/connections.py#L597

profiles.yml

dbt:
  target: dev
  outputs:
    dev:
      type: spark
      method: thrift
      host: localhost
      port: 10001
      schema: default
      auth: LDAP
      use_ssl: true

my debug progress encountered the error

> /usr/local/lib/python3.11/site-packages/thrift_sasl/__init__.py(82)open()
-> ret, chosen_mech, initial_response = self.sasl.start(self.mechanism)
(Pdb) n
AttributeError: 'SASLClient' object has no attribute 'start'

Expected Behavior

successfully establish thrift connection with enabled ssl without verifying

Steps To Reproduce

  1. setup spark thrift server:
  • enable TCP transport mode
  • enable SSL
<configuration>
    <!-- Enable Thrift Server and set the transport mode to TCP -->
    <property>
        <name>hive.server2.thrift.bind.host</name>
        <value>0.0.0.0</value>
        <description>Bind host for the Hive Thrift Server.</description>
    </property>

    <property>
        <name>hive.server2.thrift.port</name>
        <value>10001</value>
        <description>Port for the Hive Thrift Server.</description>
    </property>

    <property>
        <name>hive.server2.transport.mode</name>
        <value>TCP</value>
        <description>Transport mode for the Hive Thrift Server. Set to TCP.</description>
    </property>

    <property>
        <name>hive.server2.thrift.server.single.session</name>
        <value>true</value>
        <description>Enable single session mode for the Thrift Server.</description>
    </property>

    <!-- SSL Configuration -->
    <property>
        <name>hive.server2.use.SSL</name>
        <value>true</value>
        <description>Enable SSL for Hive Thrift Server.</description>
    </property>

    <property>
        <name>hive.server2.ssl.protocol</name>
        <value>TLSv1.2</value>
        <description>SSL protocol for Hive Thrift Server.</description>
    </property>

    <property>
        <name>hive.server2.ssl.enabledAlgorithms</name>
        <value>TLSv1.2</value>
        <description>SSL algorithms for Hive Thrift Server.</description>
    </property>

    <property>
        <name>hive.server2.keystore.path</name>
        <value>/opt/bitnami/spark/keystore.jks</value>
        <description>Path to the keystore file for SSL.</description>
    </property>

    <property>
        <name>hive.server2.keystore.password</name>
        <value>password</value>
        <description>Password for the keystore file.</description>
    </property>

    <property>
        <name>hive.server2.truststore.path</name>
        <value>/opt/bitnami/spark/truststore.jks</value>
        <description>Path to the truststore file for SSL.</description>
    </property>

    <property>
        <name>hive.server2.truststore.password</name>
        <value>password</value>
        <description>Password for the truststore file.</description>
    </property>

    <!-- Authentication Configuration -->
    <property>
        <name>hive.server2.authentication</name>
        <value>NONE</value>
        <description>Authentication mechanism for Hive Thrift Server.</description>
    </property>

    <property>
        <name>hive.server2.authentication.kerberos.mode</name>
        <value>none</value>
        <description>Kerberos mode for Hive Thrift Server (none).</description>
    </property>

    <property>
        <name>hive.server2.authentication.provider</name>
        <value>CONFIG</value>
        <description>Password authentication provider.</description>
    </property>

    <property>
        <name>hive.server2.authentication.config.username</name>
        <value>testuser</value>
        <description>Username for PLAIN authentication.</description>
    </property>

    <property>
        <name>hive.server2.authentication.config.password</name>
        <value>testpassword</value>
        <description>Password for PLAIN authentication.</description>
    </property>
</configuration>
  1. setup dbt profile
dbt:
  target: dev
  outputs:
    dev:
      type: spark
      method: thrift
      host: localhost
      port: 10001
      schema: default
      auth: LDAP
      use_ssl: true

Relevant log output

No response

Environment

- OS: Ubuntu 22.04
- Python: 3.11
- dbt-core: 1.8.5
- dbt-spark: 1.8.0

Additional Context

No response
@ntnhaatj ntnhaatj added type:bug Something isn't working as documented triage:product In Product's queue labels Aug 31, 2024
@ntnhaatj ntnhaatj linked a pull request Aug 31, 2024 that will close this issue
Open
4 tasks
@amychen1776
Copy link
Contributor

@ntnhaatj has this issue been happening or have you been using this adapter and it just came up? Trying to understand if this is a regression

@amychen1776 amychen1776 added triage:awaiting-response Awaiting a response from the reporter and removed triage:product In Product's queue labels Sep 5, 2024
@ntnhaatj
Copy link
Author

ntnhaatj commented Sep 6, 2024

@amychen1776 Not sure if this is a regression or not. We're currently integrating this adapter to work with our SSL-enabled server. It was working fine before without SSL enabled.

@github-actions github-actions bot added triage:product In Product's queue and removed triage:awaiting-response Awaiting a response from the reporter labels Sep 6, 2024
@amychen1776
Copy link
Contributor

Thank you @ntnhaatj for the information!

@ntnhaatj
Copy link
Author

@amychen1776 I’ve pushed a fix in PR dbt-labs/dbt-spark#1095 which only fix the compatibility of using pure-sasl

there’s a better way to support adding certificates to the Thrift connection instead of bypassing certificate verification. I’ll create another PR later to add that feature, if you don't mind.

@mikealfare mikealfare added the pkg:dbt-spark Issue affects dbt-spark label Jan 13, 2025
@mikealfare mikealfare transferred this issue from dbt-labs/dbt-spark Jan 13, 2025
colin-rogers-dbt pushed a commit that referenced this issue Feb 3, 2025
@talhasenSC
feat: Implement insert condition for merge strategy. (#475)
d2bd2c8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
pkg:dbt-spark Issue affects dbt-spark triage:product In Product's queue type:bug Something isn't working as documented
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: update to use pyhive function to get compatible sasl ntnhaatj/dbt-spark
3 participants
@mikealfare @ntnhaatj @amychen1776