Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Darwin passthru VM tests with remote builders #439

Open
roberth opened this issue Jun 6, 2024 · 1 comment
Open

Fix Darwin passthru VM tests with remote builders #439

roberth opened this issue Jun 6, 2024 · 1 comment

Comments

@roberth
Copy link
Member

roberth commented Jun 6, 2024

Is your feature request related to a problem? Please describe.

Since NixOS tests can run on Darwin since a few months, OfBorg is trying to run them, but these tests rely on native Linux builds, which should be performed by a remote (or local VM) builder.

Describe the solution you'd like

Ideally, all OfBorg nodes can delegate a few builds to each other node, but the remote build implementation does not currently support cyclic remote builder setups.

As an alternative, we could configure the OfBorg cluster in two layers, groups of machines, where the nodes in the top layer run the OfBorg evaluations, and the lower layer only performs builds.
The top layer would not receive remote build requests from other nodes in the top layer, and the lower layer only receives builds for the intended derivation system, making this setup acyclic.

If the top layer also performs builds for their native system, that also works, but is perhaps less effective at distributing work, as they can only be allowed to build locally evaluated derivations.

Describe alternatives you've considered

Make the passthru tests use cross compilation. I don't think this is desirable; see NixOS/nixpkgs#317651 (review)

Additional context

The docker-tools-cross VM test also has a multi-system derivation graph that requires remote builders to work. (Cross compile an image on one system, run on another)

@vcunat
Copy link
Member

vcunat commented Jun 6, 2024

This is @cole-h's territory I think. (but I'm not sure this will get high enough relative priority)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants