Can't build project with a git-based dependency using path in its workspace

[kalebpace]

Hello!

I've been trying to build a small hello world type project which uses the jj-cli crate from the Jujutsu (JJ) project. This crate depends on a workspace crate called jj-lib which is specified via a relative path within the root workspace.

When I specify the dependency like this jj-cli = { git = "https://github.com/martinvonz/jj.git" }, I get the following error:

error: failed to run custom build command for `jj-cli v0.20.0 (https://github.com/martinvonz/jj.git?rev=da7b1e9e183b45921bbd938068be9c2ae0d301ca#da7b1e9e)`
       >
       > Caused by:
       >   process didn't exit successfully: `/private/tmp/nix-build-jjui-deps-0.1.0.drv-6/source/target/release/build/jj-cli-86aae2e7caadcade/build-script-build` (exit status: 101)
       >   --- stderr
       >   thread 'main' panicked at /nix/store/3zp1k9kkcfymddjp10b9m86y985r6jnx-vendor-cargo-deps/dc64e7c94be08f133c878471bef8afa28413fc0fa105f3389d323f65e1a0b7df/jj-cli-0.20.0/build.rs:30:10:
       >   called `Result::unwrap()` on an `Err` value: CargoMetadata { stderr: "error: failed to get `jj-lib` as a dependency of package `jj-cli v0.20.0 (/nix/store/mxyibc96ylxqa09b80cgg3fyavmcgb13-cargo-git/jj-cli-0.20.0)`\n\nCaused by:\n  failed to load source for dependency `jj-lib`\n\nCaused by:\n  Unable to update /nix/store/mxyibc96ylxqa09b80cgg3fyavmcgb13-cargo-git/jj-cli-0.20.0/lib\n\nCaused by:\n  failed to read `/nix/store/mxyibc96ylxqa09b80cgg3fyavmcgb13-cargo-git/jj-cli-0.20.0/lib/Cargo.toml`\n\nCaused by:\n  No such file or directory (os error 2)\n" }
       >   note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
       > warning: build failed, waiting for other jobs to finish...

Which leads me to think that this sub-crate is not being copied into the vendor directory as I might expect. This is where the error points to.

If I specify the dependency with a version like jj-cli = "0.20.0", then I get a different error:

error: builder for '/nix/store/2qcwampdm5gffrc450daaym3f8cgcsx5-jjui-deps-0.1.0.drv' failed with exit code 101;
       last 10 log lines:
       >    Compiling gpui_macros v0.1.0 (https://github.com/zed-industries/zed.git?rev=d682594c4aff3a827f54a4aaffba4704bb61d684#d682594c)
       > error: failed to run custom build command for `jj-cli v0.20.0`
       >
       > Caused by:
       >   process didn't exit successfully: `/private/tmp/nix-build-jjui-deps-0.1.0.drv-6/source/target/release/build/jj-cli-b052007599a50185/build-script-build` (exit status: 101)
       >   --- stderr
       >   thread 'main' panicked at /nix/store/czm5gjs6v867fr7k0vlvp9n0v09nap1s-vendor-cargo-deps/c19b7c6f923b580ac259164a89f2577984ad5ab09ee9d583b888f934adbbe8d0/jj-cli-0.20.0/build.rs:30:10:
       >   called `Result::unwrap()` on an `Err` value: CargoMetadata { stderr: "error: failed to select a version for the requirement `clap = \"^4.5.13\"` (locked to 4.5.13)\ncandidate versions found which didn't match: 4.5.16\nlocation searched: directory source `/nix/store/czm5gjs6v867fr7k0vlvp9n0v09nap1s-vendor-cargo-deps/c19b7c6f923b580ac259164a89f2577984ad5ab09ee9d583b888f934adbbe8d0` (which is replacing registry `crates-io`)\nrequired by package `jj-cli v0.20.0 (/nix/store/incdl5f8af7mzlh9n7mbc4d5k7pridqn-cargo-package-jj-cli-0.20.0)`\nperhaps a crate was updated and forgotten to be re-vendored?\n" }
       >   note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
       > warning: build failed, waiting for other jobs to finish...

Which I am also not sure how to handle either. Though am primarily interested in getting the git-based dependency working.

I am new to using crane, so unsure how to proceed from here. Any guidance would be much appreciated!

[ipetkov]

Hi @kalebpace thanks for the report! Is your Cargo.lock fully up to date? I'd recommend running any sort of cargo {build,check,clippy} (or just plain old cargo update) and staging any changes that show up in Cargo.lock. Otherwise if the Cargo.lock file is fully up to date and building outside of Nix works I'd be curious to look at a(n ideally minimal) reproduction flake!

[kalebpace]

Hey @ipetkov, thanks for taking time to look at this! 🙏

I've regenerated Cargo.lock several times to ensure its being generated by the same toolchain defined in the flake and up-to-date. Same errors.

It does build outside of Nix, as well as within the dev shell provided by crane.

If you'd like to take a look, the project is fully up-to-date here: https://github.com/kalebpace/jj-gui

[ipetkov]

Oh I see, the build script is effectively trying to run cargo metadata and find the path to its relative member! Yes unfortunately this won't fly because cargo expects each crate to be vendored differently than the way that "raw" git-dependencies happen to work...

Please take a look at the following FAQ which should explain the situation further and suggest some workarounds: https://github.com/ipetkov/crane/blob/d995bc5e2d5853a72367d6e10df0b756f38a3aff/docs/faq/git-dep-cannot-find-relative-path.md

(We've had questions about this situation come up before and somehow I hadn't actually documented it until now; better late than never, hope it helps!)

[kalebpace]

Amazing, makes sense! Thanks so much for confirming this and adding it to the FAQ. Will reach out to the JJ project to try resolving this upstream 🙂