Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitManager._checkout() fails in windows if repo has a symlink in it #417

Open
andyk opened this issue Jun 28, 2022 · 0 comments
Open

GitManager._checkout() fails in windows if repo has a symlink in it #417

andyk opened this issue Jun 28, 2022 · 0 comments

Comments

@andyk
Copy link
Contributor

andyk commented Jun 28, 2022

Our windows tests started failing when the SB3 repo (the agentos freeze test uses) added a symlink (on June 22 2022) to their repo: https://github.com/mgbellemare/Arcade-Learning-Environment/blob/master/src/python/roms/md5.txt

The file is getting created in my windows 11 filesystem but it is not a valid symlink
invalid_file_screenshot

The error is in Dulwich's porcelain.py file:
https://github.com/jelmer/dulwich/blob/bbf100b63fb00ebb4a1d0ca698b1abe337611bae/dulwich/porcelain.py#L230 in path_to_tree_path() in a call to Path.resolve() on the symlinked file.

The full failed test stack trace is:


    def test_cli_freeze(cli_runner, tmpdir):
        run_args = ["sb3_agent", "-f"]
        run_kwargs = {"--registry-file": str(SB3_AGENT_DIR / "components.yaml")}
>       run_test_command(
            cli_runner, freeze, cli_args=run_args, cli_kwargs=run_kwargs
        )

C:\Users\andyk\Development\agentos\tests\test_core.py:57: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
C:\Users\andyk\Development\agentos\tests\utils.py:32: in run_test_command
    result = cli_runner.invoke(
C:\Users\andyk\Development\agentos\tests\conftest.py:26: in wrapper
    result = f(*args, **kwargs)
C:\Users\andyk\Development\agentos\windows_dev\lib\site-packages\click\testing.py:408: in invoke
    return_value = cli.main(args=args or (), prog_name=prog_name, **extra)
C:\Users\andyk\Development\agentos\windows_dev\lib\site-packages\click\core.py:1062: in main
    rv = self.invoke(ctx)
C:\Users\andyk\Development\agentos\windows_dev\lib\site-packages\click\core.py:1404: in invoke
    return ctx.invoke(self.callback, **ctx.params)
C:\Users\andyk\Development\agentos\windows_dev\lib\site-packages\click\core.py:763: in invoke
    return __callback(*args, **kwargs)
C:\Users\andyk\Development\agentos\agentos\cli.py:269: in freeze
    frozen_reg = module.freeze(force=force).to_registry()
C:\Users\andyk\Development\agentos\pcs\instance_manager.py:46: in freeze
    find_and_replace_leaves(
C:\Users\andyk\Development\agentos\pcs\utils.py:220: in find_and_replace_leaves
    leaf_replace(data_struct, leaf_list, replace_fn)
C:\Users\andyk\Development\agentos\pcs\utils.py:156: in leaf_replace
    data_struct[leaf_list[0]] = replacement_fn(next_inner)
C:\Users\andyk\Development\agentos\pcs\instance_manager.py:49: in <lambda>
    lambda x: x.freeze(force),
C:\Users\andyk\Development\agentos\pcs\class_manager.py:86: in freeze
    self_copy.module = self.module.freeze(force)
C:\Users\andyk\Development\agentos\pcs\module_manager.py:184: in freeze
    return self.to_versioned_module(force)
C:\Users\andyk\Development\agentos\pcs\module_manager.py:156: in to_versioned_module
    repo_url, version = self.repo.get_version_from_git(
C:\Users\andyk\Development\agentos\pcs\repo.py:73: in get_version_from_git
    full_path = self.get_local_file_path(file_path, version)
C:\Users\andyk\Development\agentos\pcs\repo.py:123: in get_local_file_path
    local_repo_path = self.get_local_repo_dir(version)
C:\Users\andyk\Development\agentos\pcs\repo.py:115: in get_local_repo_dir
    local_repo_path = self.GIT.clone_repo(
C:\Users\andyk\Development\agentos\pcs\git_manager.py:295: in clone_repo
    self._clone_repo(
C:\Users\andyk\Development\agentos\pcs\git_manager.py:317: in _clone_repo
    self._checkout_version(clone_destination, version)
C:\Users\andyk\Development\agentos\pcs\git_manager.py:331: in _checkout_version
    self._checkout(repo=repo, target=treeish, force=True)
C:\Users\andyk\Development\agentos\pcs\git_manager.py:473: in _checkout
    for file in porcelain.get_untracked_paths(
C:\Users\andyk\Development\agentos\windows_dev\lib\site-packages\dulwich\porcelain.py:1283: in get_untracked_paths
    ip = path_to_tree_path(basepath, ap)
C:\Users\andyk\Development\agentos\windows_dev\lib\site-packages\dulwich\porcelain.py:233: in path_to_tree_path
    resolved_path = path.resolve()
C:\Python39\lib\pathlib.py:1204: in resolve
    s = self._flavour.resolve(self, strict=strict)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pathlib._WindowsFlavour object at 0x0000018C2CDBFA30>
path = WindowsPath('C:/Users/andyk/.agentos/cache/repos_cache/mgbellemare/Arcade-Learning-Environment/db3728264f382402120913d76c4fa0dc320ef59f/src/python/roms/md5.txt')
strict = False

    def resolve(self, path, strict=False):
        s = str(path)
        if not s:
            return os.getcwd()
        previous_s = None
        if _getfinalpathname is not None:
            if strict:
                return self._ext_to_normal(_getfinalpathname(s))
            else:
                tail_parts = []  # End of the path after the first one not found
                while True:
                    try:
>                       s = self._ext_to_normal(_getfinalpathname(s))
E                       OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: 'C:\\Users\\andyk\\.agentos\\cache\\repos_cache\\mgbellemare\\Arcade-Learning-Environment\\db3728264f382402120913d76c4fa0dc320ef59f\\src\\python\\roms\\md5.txt'

C:\Python39\lib\pathlib.py:205: OSError
@nickjalbert nickjalbert mentioned this issue Jun 30, 2022
nickjalbert added a commit to nickjalbert/agentos that referenced this issue Jun 30, 2022
nickjalbert added a commit to nickjalbert/agentos that referenced this issue Jul 1, 2022
nickjalbert added a commit to nickjalbert/agentos that referenced this issue Jul 1, 2022
nickjalbert added a commit to nickjalbert/agentos that referenced this issue Jul 1, 2022
nickjalbert added a commit to nickjalbert/agentos that referenced this issue Jul 1, 2022
nickjalbert added a commit to nickjalbert/agentos that referenced this issue Jul 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant