Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: nix flake #5267

Draft
wants to merge 18 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .github/workflows/nix.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: nix
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
name: nix
name: nix
# Cancel workflow if there is a new change to the branch.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lemmih what do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will update PR in a bit.

on:
workflow_dispatch:
merge_group:
pull_request:
branches:
- main
push:
branches:
- main

jobs:
build-nix-package:
strategy:
matrix:
os: [ubuntu-24.04, ubuntu-24.04-arm, macos-14]
runs-on: ${{ matrix.os }}
permissions:
contents: read
id-token: write
steps:
- uses: actions/checkout@v3
- uses: DeterminateSystems/nix-installer-action@main
with:
determinate: true
- uses: DeterminateSystems/flakehub-cache-action@main
- run: nix build --dry-run
- run: nix build
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,15 @@ documentation in the [Forest Book]. Keep in mind that the `latest` tag is the la
stable release. If you want to use the current development build, use the `edge`
tag.

## Run with Nix (as a flake)

```
nix profile install github:ChainSafe/forest
```

This will make the `forest`, `forest-cli`, `forest-tool`, and `forest-wallet`
commands available in your shell.

## Dependencies

- Rust (toolchain version is specified in `rust-toolchain.toml`)
Expand Down
92 changes: 92 additions & 0 deletions README.nix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Using Nix with Forest

This guide will help you get started with building and installing Forest using
Nix.

## Installing Nix

The recommended way to install Nix is using Determinate Systems' Nix installer:

1. Run the following command in your terminal:
```bash
curl -fsSL https://install.determinate.systems/nix | sh -s -- install --determinate
```

2. After installation, restart your shell or run:
```bash
. /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh
```

See [Determinate Systems' Nix installation
guide](https://docs.determinate.systems/getting-started/individuals) for more
information.

## Building Forest with Flakes

Forest uses Nix flakes for reproducible builds. To build Forest:

1. Clone the Forest repository
2. Run the following command in the repository root:
```bash
nix build
```

This will build Forest and all its dependencies in a reproducible environment.

## Installing Forest

To install Forest directly from the repository:

```bash
nix profile install .
```

This will make the following Forest commands available in your shell:
- `forest` - The main Forest daemon
- `forest-cli` - Command line interface for interacting with Forest
- `forest-tool` - Utility tools for Forest
- `forest-wallet` - Forest wallet management

You can now run any of these commands directly from your shell.

## Upgrading Forest

To upgrade Forest to the latest version:

```bash
nix profile upgrade forest
```

## Removing Forest

To remove Forest from your system:

```bash
nix profile remove forest
```

You can also list your current profile installations using:

```bash
nix profile list
```

## Troubleshooting

If you encounter any issues with Nix:
- Make sure you have the latest version of Nix installed
- Try running `nix-collect-garbage` if you're running low on disk space

### Build Environment Limitations

- Nix builds occur in a read-only environment. This means that build scripts
cannot generate or modify source files during the build process
- If your build requires non-Rust files (e.g., JSON, proto files, or other
assets), they must be explicitly listed in the flake.nix file
- If you see errors about missing files during the build, check that they are
properly included in the flake's source inputs

Example error messages you might encounter:
- "Permission denied" when trying to write files during build
- "File not found" for non-Rust files that aren't explicitly included in the
flake
21 changes: 21 additions & 0 deletions docs/docs/users/getting_started/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,27 @@ forest-filecoin 0.19.0+git.671c30c
</TabItem>
<TabItem value="docker" label="Docker">

<h3>Nix Flake</h3>

To install Forest as a Nix flake:

```shell
nix profile install github:ChainSafe/forest
```

This will make the `forest`, `forest-cli`, `forest-tool`, and `forest-wallet`
commands available in your shell.

```shell
forest --version
```

Sample output:

```console
forest-filecoin 0.19.0+git.671c30c
```

<h3>Images</h3>

Images are available via Github Container Registry:
Expand Down
98 changes: 98 additions & 0 deletions flake.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

130 changes: 130 additions & 0 deletions flake.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
{
description = "Forest - A Rust implementation of Filecoin";

inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.11";
crane.url = "github:ipetkov/crane";
flake-utils.url = "github:numtide/flake-utils";
rust-overlay = {
url = "github:oxalica/rust-overlay";
inputs.nixpkgs.follows = "nixpkgs";
};
};

outputs = {
self,
nixpkgs,
crane,
flake-utils,
rust-overlay,
...
}:
flake-utils.lib.eachDefaultSystem (system: let
pkgs = import nixpkgs {
inherit system;
overlays = [(import rust-overlay)];
};

rustToolchain = pkgs.rust-bin.fromRustupToolchainFile ./rust-toolchain.toml;
craneLib = (crane.mkLib pkgs).overrideToolchain rustToolchain;

# This is a hack to get the f3-sidecar vendor files into the build directory
f3-sidecar-files = pkgs.buildGoModule {
pname = "f3-sidecar-files";
version = "0.1.0";
src = pkgs.lib.cleanSourceWith {
src = ./f3-sidecar;
};
buildPhase = ''
mkdir -p $out
cp -r . $out
'';
vendorHash = "sha256-Pe3bgBZr8pdn2XVOQRvEjvnXldw7N1ehv0zY51kGSSk=";
};

src = pkgs.lib.cleanSourceWith {
src = ./.;
filter = path: type:
(craneLib.filterCargoSources path type)
|| (type == "regular" && pkgs.lib.hasSuffix ".stpl" path)
|| (type == "regular" && pkgs.lib.hasSuffix ".json" path)
|| (type == "regular" && pkgs.lib.hasSuffix ".car" path)
|| (type == "regular" && pkgs.lib.hasSuffix ".txt" path)
|| (pkgs.lib.hasInfix "/build/" path)
|| (pkgs.lib.hasInfix "/f3-sidecar/" path)
|| (pkgs.lib.hasInfix "/test-snapshots/" path);
};

# Common arguments can be set here to avoid repeating them later
commonArgs = {
inherit src;

buildInputs = with pkgs; [
# Add runtime dependencies here
];

nativeBuildInputs = with pkgs; [
# Add build-time dependencies here
go # For rust2go compilation
clang # For C/C++ compilation
];

doCheck = false;
};

# Build *just* the cargo dependencies, so we can reuse
# all of that work (e.g. via cachix) when running in CI
cargoArtifacts = craneLib.buildDepsOnly commonArgs;

# Build the actual crate itself, reusing the dependency
# artifacts from above.
forest = craneLib.buildPackage (commonArgs
// {
inherit cargoArtifacts;
preConfigure = ''
export GOCACHE=$(mktemp -d)
export GOMODCACHE=$(mktemp -d)
'';
preBuild = ''
# Copy f3-sidecar files into the build directory
cp -r ${f3-sidecar-files}/vendor f3-sidecar/
'';
LIBCLANG_PATH = "${pkgs.llvmPackages.libclang.lib}/lib";
# Environment variables needed for the build
# FOREST_F3_SIDECAR_FFI_BUILD_OPT_OUT = "1";
});
in {
checks = {
# Build the crate as part of `nix flake check` for convenience
inherit forest;
};

packages.default = forest;

apps = let
binaries = ["forest" "forest-cli" "forest-tool" "forest-wallet"];
mkBinApp = name:
flake-utils.lib.mkApp {
drv = forest;
inherit name;
};
in
{
default = mkBinApp "forest";
}
// builtins.listToAttrs (map (name: {
inherit name;
value = mkBinApp name;
})
binaries);

devShells.default = pkgs.mkShell {
inputsFrom = builtins.attrValues self.checks.${system};

# Additional dev-shell environment variables can be set directly
shellHook = ''
echo "Forest development shell"
'';
};
});
}
Loading