Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate netboot.gsc.io ("netboot-foundation" EQM host) #416

Open
delroth opened this issue Apr 18, 2024 · 4 comments
Open

Deprecate netboot.gsc.io ("netboot-foundation" EQM host) #416

delroth opened this issue Apr 18, 2024 · 4 comments
Assignees

Comments

@delroth
Copy link
Contributor

delroth commented Apr 18, 2024

We run a c3.medium.x86 instance on packet that literally only serves static files on domain name netboot.gsc.io for the aarch64 community build box to be able to netboot.

It would be better to:

  1. Build the community build box image on Hydra, similar to the Hydra builder netboot image.
  2. Switch to using netboot.nixos.org (nix-netboot-serve).

@lheckemann suggested he might be help to pair up with me on this.

cc @cole-h

@delroth
Copy link
Contributor Author

delroth commented Apr 18, 2024

https://hydra.nixos.org/jobset/equinix-metal-builders/aarch64-community-box now builds the system for netboot.nixos.org.

@lheckemann
Copy link
Member

There's a pile of horrible hacks on https://github.com/lheckemann/aarch64-build-box which is currently running fine but is almost certainly not what we want long-term.

nix-netboot-serve does some Unusual things with the boot environment, preserving the initrd as the final root filesystem, but we don't want that here. Thus, we apply some deep hacks to have the classic NixOS stage-1 used as the initrd init (even though our initrd contains the whole system), copy the system closure into the real filesystem, and then hand over to something resembling the regular stage-2 (further hacks applied to get the system config path correctly).

We'll probably want to change nix-netboot-serve up a bit to support our use case better -- most importantly getting closer to the regular NixOS boot process, running stage-1 and stage-2 as usual. We should think about this some more at more reasonable "awake times". :)

@delroth
Copy link
Contributor Author

delroth commented Apr 19, 2024

Note: long term we probably want to converge the semi-persistence setup used on the aarch64-build-box with the one used on hydra builders (which has issues), so taking care of this with the aarch64-build-box seems like a very valuable time investment.

@delroth
Copy link
Contributor Author

delroth commented Apr 20, 2024

Filed DeterminateSystems/nix-netboot-serve#26 too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

No branches or pull requests

2 participants