Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dvc exp push fails if git connection is over a proxyjump #10683

Closed
f-fuchs opened this issue Feb 4, 2025 · 10 comments
Closed

dvc exp push fails if git connection is over a proxyjump #10683

f-fuchs opened this issue Feb 4, 2025 · 10 comments
Labels
git Related to git and git backends

Comments

@f-fuchs
Copy link

f-fuchs commented Feb 4, 2025

Description

Hey,

I have to use an ssh proxyjump to connect to my git repository. Sadly this results in all the dvc commands that rely on the remote failing.

Reproduce

If I run a normal git command everything is fine and I get prompted for the 2fa for the ssh server.

(demultiple) [fuchsfa@cerberus04:demultiple]$ git push
([email protected]) Password: 
([email protected]) Verification code: 
Enter passphrase for key 'path': 
Everything up-to-date

Sadly dvc does not prompt me for my credentials but instead just fails.

(demultiple) [fuchsfa@cerberus04:demultiple]$ dvc exp push origin -v
2025-02-04 14:31:46,636 DEBUG: v3.56.0 (pip), CPython 3.12.7 on Linux-5.15.0-131-generic-x86_64-with-glibc2.36
2025-02-04 14:31:46,637 DEBUG: command: /home/fuchsfa/demultiple/.venv/bin/dvc exp push origin -v
2025-02-04 14:31:49,347 DEBUG: git push experiment [] -> 'origin'     
2025-02-04 14:31:50,899 ERROR: Authentication failed for: 'GIT_SERVER'                                                                                                                                                                                                                                                                                                                                                                                                 
See https://dvc.org/doc/user-guide/troubleshooting#git-auth
Traceback (most recent call last):
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/scmrepo/git/backend/dulwich/asyncssh_vendor.py", line 297, in _run_command
    conn = await asyncssh.connect(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/asyncssh/connection.py", line 8865, in connect
    return await asyncio.wait_for(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/.local/share/uv/python/cpython-3.12.7-linux-x86_64-gnu/lib/python3.12/asyncio/tasks.py", line 520, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/asyncssh/connection.py", line 401, in _connect
    new_tunnel = await _open_tunnel(tunnel, options.passphrase, config)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/asyncssh/connection.py", line 375, in _open_tunnel
    conn = await connect(host, port, username=username,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/asyncssh/connection.py", line 8865, in connect
    return await asyncio.wait_for(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/.local/share/uv/python/cpython-3.12.7-linux-x86_64-gnu/lib/python3.12/asyncio/tasks.py", line 520, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/asyncssh/connection.py", line 450, in _connect
    await options.waiter
asyncssh.misc.PermissionDenied: Permission denied for user fuchsfa on host SSH_SERVER

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/repo/experiments/push.py", line 153, in _push
    results: Mapping[str, SyncStatus] = repo.scm.push_refspecs(
                                        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/scmrepo/git/__init__.py", line 307, in _backend_func
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/scmrepo/git/backend/dulwich/__init__.py", line 666, in push_refspecs
    result = client.send_pack(
             ^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dulwich/client.py", line 1239, in send_pack
    proto, unused_can_read, stderr = self._connect(b"receive-pack", path)
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dulwich/client.py", line 2191, in _connect
    con = self.ssh_vendor.run_command(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/fsspec/asyn.py", line 118, in wrapper
    return sync(self.loop, func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/fsspec/asyn.py", line 103, in sync
    raise return_result
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/fsspec/asyn.py", line 56, in _runner
    result[0] = await coro
                ^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/scmrepo/git/backend/dulwich/asyncssh_vendor.py", line 312, in _run_command
    raise AuthError(f"{username}@{host}:{port or 22}") from exc
scmrepo.exceptions.AuthError: Authentication failed for: 'GIT_SERVER'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/cli/__init__.py", line 211, in main
    ret = cmd.do_run()
          ^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/cli/command.py", line 27, in do_run
    return self.run()
           ^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/commands/experiments/push.py", line 55, in run
    result = self.repo.experiments.push(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/repo/experiments/__init__.py", line 364, in push
    return push(self.repo, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/repo/__init__.py", line 58, in wrapper
    return f(repo, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/repo/scm_context.py", line 143, in run
    return method(repo, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/repo/experiments/push.py", line 111, in push
    push_result = _push(repo, git_remote, exp_ref_set, force)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/repo/experiments/push.py", line 160, in _push
    raise GitAuthError(str(exc))  # noqa: B904
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dvc.scm.GitAuthError: Authentication failed for: 'GIT_SERVER'
See https://dvc.org/doc/user-guide/troubleshooting#git-auth

2025-02-04 14:31:50,964 DEBUG: Analytics is enabled.
2025-02-04 14:31:51,696 DEBUG: Trying to spawn ['daemon', 'analytics', '/tmp/tmp4zxtqbcl', '-v']
2025-02-04 14:31:51,715 DEBUG: Spawned ['daemon', 'analytics', '/tmp/tmp4zxtqbcl', '-v'] with pid 1400

Is there a way for dvc to prompt me? Alternatively is there a way to figure out which git commands this would run and then run them myself?
@skshetry
Copy link
Member

skshetry commented Feb 4, 2025

Where do you have your keys' setup? Are those in ~/.ssh/config?

@f-fuchs
Copy link
Author

f-fuchs commented Feb 4, 2025

I only have a ssh key for the git but the ssh server which is used as proxy requires a password and 2fa token.
Which is what breaks I think, because I never get prompted for them.

@skshetry
Copy link
Member

skshetry commented Feb 4, 2025

It could be because we don't implement password authentication. We do implement keyword-interactive authentication, and maybe the server is not using that.

@skshetry
Copy link
Member

skshetry commented Feb 4, 2025

Can you try this PR and see if fixes the issue: iterative/scmrepo#403?

You can install it as follows (in the same environment where dvc is installed).

$ pip install "scmrepo @ git+https://github.com/iterative/scmrepo.git@fix-10683"

@f-fuchs
Copy link
Author

f-fuchs commented Feb 4, 2025

installing worked fine.

(demultiple) [fuchsfa@cerberus04:demultiple]$ uv pip install "scmrepo @ git+https://github.com/iterative/scmrepo.git@fix-10683"
 Updated https://github.com/iterative/scmrepo.git (8d18813)
Resolved 27 packages in 19.64s
   Built scmrepo @ git+https://github.com/iterative/scmrepo.git@8d18813555b47707627c6fd7dbc22dad5e835730
Prepared 1 package in 10.18s
Uninstalled 1 package in 69ms
Installed 1 package in 138ms
 - scmrepo==3.3.8
 + scmrepo==3.3.11.dev1+g8d18813 (from git+https://github.com/iterative/scmrepo.git@8d18813555b47707627c6fd7dbc22dad5e835730)

but the error message is still the same i think 🤔

(demultiple) [fuchsfa@cerberus04:demultiple]$ dvc exp push origin
ERROR: Authentication failed for: 'GIT_SERVER'                                                                                                                                                                                                                                                
See https://dvc.org/doc/user-guide/troubleshooting#git-auth
(demultiple) [fuchsfa@cerberus04:demultiple]$ dvc exp push origin -v
2025-02-04 15:22:56,279 DEBUG: v3.56.0 (pip), CPython 3.12.7 on Linux-5.15.0-131-generic-x86_64-with-glibc2.36
2025-02-04 15:22:56,280 DEBUG: command: /home/fuchsfa/demultiple/.venv/bin/dvc exp push origin -v
2025-02-04 15:22:58,917 DEBUG: git push experiment [] -> 'origin'     
2025-02-04 15:23:00,343 ERROR: Authentication failed for: 'GIT_SERVER'                                                                                                                                                                                                                        
See https://dvc.org/doc/user-guide/troubleshooting#git-auth
Traceback (most recent call last):
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/scmrepo/git/backend/dulwich/asyncssh_vendor.py", line 306, in _run_command
    conn = await asyncssh.connect(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/asyncssh/connection.py", line 8865, in connect
    return await asyncio.wait_for(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/.local/share/uv/python/cpython-3.12.7-linux-x86_64-gnu/lib/python3.12/asyncio/tasks.py", line 520, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/asyncssh/connection.py", line 401, in _connect
    new_tunnel = await _open_tunnel(tunnel, options.passphrase, config)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/asyncssh/connection.py", line 375, in _open_tunnel
    conn = await connect(host, port, username=username,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/asyncssh/connection.py", line 8865, in connect
    return await asyncio.wait_for(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/.local/share/uv/python/cpython-3.12.7-linux-x86_64-gnu/lib/python3.12/asyncio/tasks.py", line 520, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/asyncssh/connection.py", line 450, in _connect
    await options.waiter
asyncssh.misc.PermissionDenied: Permission denied for user fuchsfa on host SSH_SERVER

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/repo/experiments/push.py", line 153, in _push
    results: Mapping[str, SyncStatus] = repo.scm.push_refspecs(
                                        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/scmrepo/git/__init__.py", line 307, in _backend_func
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/scmrepo/git/backend/dulwich/__init__.py", line 666, in push_refspecs
    result = client.send_pack(
             ^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dulwich/client.py", line 1239, in send_pack
    proto, unused_can_read, stderr = self._connect(b"receive-pack", path)
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dulwich/client.py", line 2191, in _connect
    con = self.ssh_vendor.run_command(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/fsspec/asyn.py", line 118, in wrapper
    return sync(self.loop, func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/fsspec/asyn.py", line 103, in sync
    raise return_result
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/fsspec/asyn.py", line 56, in _runner
    result[0] = await coro
                ^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/scmrepo/git/backend/dulwich/asyncssh_vendor.py", line 321, in _run_command
    raise AuthError(f"{username}@{host}:{port or 22}") from exc
scmrepo.exceptions.AuthError: Authentication failed for: 'GIT_SERVER'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/cli/__init__.py", line 211, in main
    ret = cmd.do_run()
          ^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/cli/command.py", line 27, in do_run
    return self.run()
           ^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/commands/experiments/push.py", line 55, in run
    result = self.repo.experiments.push(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/repo/experiments/__init__.py", line 364, in push
    return push(self.repo, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/repo/__init__.py", line 58, in wrapper
    return f(repo, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/repo/scm_context.py", line 143, in run
    return method(repo, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/repo/experiments/push.py", line 111, in push
    push_result = _push(repo, git_remote, exp_ref_set, force)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/repo/experiments/push.py", line 160, in _push
    raise GitAuthError(str(exc))  # noqa: B904
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dvc.scm.GitAuthError: Authentication failed for: 'GIT_SERVER'
See https://dvc.org/doc/user-guide/troubleshooting#git-auth

2025-02-04 15:23:00,413 DEBUG: Analytics is enabled.
2025-02-04 15:23:01,269 DEBUG: Trying to spawn ['daemon', 'analytics', '/tmp/tmpzmolmu4_', '-v']
2025-02-04 15:23:01,296 DEBUG: Spawned ['daemon', 'analytics', '/tmp/tmpzmolmu4_', '-v'] with pid 21533

@skshetry
Copy link
Member

skshetry commented Feb 4, 2025 •
edited
Loading

asyncssh should support ProxyJump. I see a similar issue posted #7800 a few years ago, but was closed.

It's difficult for me to say what's happening. If you can debug, that'd be great.

(It could be that we need to set it up properly: ronf/asyncssh#307).

@f-fuchs
Copy link
Author

f-fuchs commented Feb 4, 2025 •
edited
Loading

so I replaced ProxyJump with ProxyCommand ssh -W %h:%p in my .config file and I get a different error now 😁

dvc exp push origin -v
2025-02-04 15:57:06,739 DEBUG: v3.56.0 (pip), CPython 3.12.7 on Linux-5.15.0-131-generic-x86_64-with-glibc2.36
2025-02-04 15:57:06,739 DEBUG: command: /home/fuchsfa/demultiple/.venv/bin/dvc exp push origin -v
2025-02-04 15:57:09,264 DEBUG: git push experiment [] -> 'origin'     
2025-02-04 15:57:10,448 ERROR: unexpected error - Connection lost                                                                                                                                                                                                                                                   
Traceback (most recent call last):
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/cli/__init__.py", line 211, in main
    ret = cmd.do_run()
          ^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/cli/command.py", line 27, in do_run
    return self.run()
           ^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/commands/experiments/push.py", line 55, in run
    result = self.repo.experiments.push(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/repo/experiments/__init__.py", line 364, in push
    return push(self.repo, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/repo/__init__.py", line 58, in wrapper
    return f(repo, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/repo/scm_context.py", line 143, in run
    return method(repo, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/repo/experiments/push.py", line 111, in push
    push_result = _push(repo, git_remote, exp_ref_set, force)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dvc/repo/experiments/push.py", line 153, in _push
    results: Mapping[str, SyncStatus] = repo.scm.push_refspecs(
                                        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/scmrepo/git/__init__.py", line 307, in _backend_func
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/scmrepo/git/backend/dulwich/__init__.py", line 666, in push_refspecs
    result = client.send_pack(
             ^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dulwich/client.py", line 1239, in send_pack
    proto, unused_can_read, stderr = self._connect(b"receive-pack", path)
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/dulwich/client.py", line 2191, in _connect
    con = self.ssh_vendor.run_command(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/fsspec/asyn.py", line 118, in wrapper
    return sync(self.loop, func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/fsspec/asyn.py", line 103, in sync
    raise return_result
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/fsspec/asyn.py", line 56, in _runner
    result[0] = await coro
                ^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/scmrepo/git/backend/dulwich/asyncssh_vendor.py", line 306, in _run_command
    conn = await asyncssh.connect(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/asyncssh/connection.py", line 8865, in connect
    return await asyncio.wait_for(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fuchsfa/.local/share/uv/python/cpython-3.12.7-linux-x86_64-gnu/lib/python3.12/asyncio/tasks.py", line 520, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/fuchsfa/demultiple/.venv/lib/python3.12/site-packages/asyncssh/connection.py", line 450, in _connect
    await options.waiter
asyncssh.misc.ConnectionLost: Connection lost

2025-02-04 15:57:10,594 DEBUG: link type reflink is not available ([Errno 95] no more link types left to try out)
2025-02-04 15:57:10,595 DEBUG: Removing '/home/fuchsfa/.Zudk8btpNAwo7jfDjZsIUQ.tmp'
2025-02-04 15:57:10,597 DEBUG: Removing '/home/fuchsfa/.Zudk8btpNAwo7jfDjZsIUQ.tmp'
2025-02-04 15:57:10,600 DEBUG: Removing '/home/fuchsfa/.Zudk8btpNAwo7jfDjZsIUQ.tmp'
2025-02-04 15:57:10,602 DEBUG: Removing '/home/fuchsfa/demultiple/.dvc/cache/files/md5/.woa4ld79ve4X8gLLXYMxuw.tmp'
2025-02-04 15:57:10,632 DEBUG: Version info for developers:
DVC version: 3.56.0 (pip)
-------------------------
Platform: Python 3.12.7 on Linux-5.15.0-131-generic-x86_64-with-glibc2.36
Subprojects:
        dvc_data = 3.16.7
        dvc_objects = 5.1.0
        dvc_render = 1.0.2
        dvc_task = 0.40.2
        scmrepo = 3.3.11.dev1+g8d18813
Supports:
        http (aiohttp = 3.11.2, aiohttp-retry = 2.9.1),
        https (aiohttp = 3.11.2, aiohttp-retry = 2.9.1)
Config:
        Global: /home/fuchsfa/.config/dvc
        System: /etc/xdg/dvc
Cache types: hardlink, symlink
Cache directory: beegfs on beegfs_nodev
Caches: local
Remotes: local
Workspace directory: beegfs on beegfs_nodev
Repo: dvc, git
Repo.site_cache_dir: /var/tmp/dvc/repo/21a362f2abe22b86be5a4df3cdf01edf

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2025-02-04 15:57:10,638 DEBUG: Analytics is enabled.
2025-02-04 15:57:11,417 DEBUG: Trying to spawn ['daemon', 'analytics', '/tmp/tmpuep1tmos', '-v']
2025-02-04 15:57:11,436 DEBUG: Spawned ['daemon', 'analytics', '/tmp/tmpuep1tmos', '-v'] with pid 36345

still no prompt though

@shcheklein shcheklein added triage Needs to be triaged fs: ssh Related to the SSH filesystem A: data-sync Related to dvc get/fetch/import/pull/push labels Feb 6, 2025
@skshetry skshetry added git Related to git and git backends and removed triage Needs to be triaged fs: ssh Related to the SSH filesystem A: data-sync Related to dvc get/fetch/import/pull/push labels Feb 6, 2025
@f-fuchs
Copy link
Author

f-fuchs commented Feb 6, 2025

turns out there was a way to get an ssh key for the proxyjump server and that seems to work. 👍
so the problem was the probably the password prompt not the proxyjump

@skshetry
Copy link
Member

skshetry commented Feb 6, 2025

Hey @f-fuchs. I would recommend trying with plain asyncssh.connect() to connect to the server, and then opening a bug report on https://github.com/ronf/asyncssh with your findings.

I don't see a way to prompt for password during proxyjump in asyncssh (maybe I did not look in the right place). I also see that we don't support prompting for passphrase for the key during proxyjump either.

@skshetry
Copy link
Member

skshetry commented Feb 6, 2025 •
edited
Loading

Also, note that, for Git operations, we use two different clients: asyncssh and OpenSSH cli.
We do this for performance reasons. asyncssh does not support everything OpenSSH does, and we try to determine which clients to use by parsing a ~/.ssh/config file. If there are config that asyncssh does not know, we fallback to using ssh cli based client.

So, if you have a complicated setup, a good workaround here could be to add some fields that are not supported by asyncssh in your config. One config that I add is AddKeysToAgent yes. You may find supported config options here: https://asyncssh.readthedocs.io/en/latest/api.html#supported-client-config-options. 😄

@f-fuchs f-fuchs closed this as completed Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
git Related to git and git backends
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@shcheklein @skshetry @f-fuchs