Building Rust With Nix One Compile Step at a Time

Intro #

In the gonixgo post I built Go with Nix one package per derivation, with the module hashes worked out while Nix evaluates. Since then I have done the same for Rust. It is called rostnix.

Rust does not have Go’s hash problem. Cargo.lock has a SHA-256 for every crate, and nixpkgs reads it. With cargoLock.lockFile = ./Cargo.lock buildRustPackage needs no hash, and with allowBuiltinFetchGit = true not even for git dependencies. rostnix itself is built that way.

The rebuilds are the same problem as with Go. buildRustPackage runs cargo build in a single derivation, so after a one-line change every dependency is compiled again.

Other Builders #

crane splits the build in two, the dependencies in one derivation and your own crates in another. That helps a lot while you only edit your own code. Touch Cargo.lock and all dependencies are built again, though, and in a workspace all of your crates are compiled again whenever one of them changes.

crate2nix gives every crate its own derivation. It describes the build in a generated Cargo.nix, which you either check in or generate during evaluation with import-from-derivation, and it works out in Nix which features each crate is built with.

cargo-nix-plugin does the same without the generated file. It is a Nix plugin that adds builtins.resolveCargoWorkspace to the evaluator. That reads Cargo.lock and the registry index, works out the crate graph and the features itself, and hands them to its own version of nixpkgs’ buildRustCrate. It is the trade go2nix makes for Go: the plugin has to be built for the exact Nix that loads it, 2.30 or newer, on every machine that evaluates the project.

I wanted something like cargo-nix-plugin that works with the Nix that is already installed. And I wanted cargo itself to plan the build, down to single compile steps rather than whole crates. There is a section further down on how the two differ.

The Flake #

Here it is for a checkout of ripgrep, which is the example for the rest of this post. It looks like the flake for gonixgo, with mkRustEnv and buildRustApplication in place of the Go functions:

{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-26.05";
  inputs.rostnix.url = "github:draganm/rostnix";

  outputs = { nixpkgs, rostnix, ... }:
    let
      system = "aarch64-darwin";
      pkgs = nixpkgs.legacyPackages.${system};
      rustEnv = rostnix.lib.mkRustEnv { inherit pkgs; };
    in {
      packages.${system}.default = rustEnv.buildRustApplication {
        pname = "ripgrep";
        src = ./.;
      };
    };
}

The build command is the same as well:

nix build --option allow-unsafe-native-code-during-evaluation true

There is a section about the option further down.

Cargo’s Plan #

As in gonixgo, a small program that Nix builds during evaluation runs through builtins.exec and prints the build as Nix code. Instead of go list it asks cargo:

RUSTC_BOOTSTRAP=1 cargo build --unit-graph -Z unstable-options --locked

--unit-graph is unstable, and RUSTC_BOOTSTRAP=1 lets a stable cargo use it anyway. It prints the plan and builds nothing. A unit is one step of that plan: one rustc invocation, or one run of a build script (build.rs). A crate with a build script is three units. In ripgrep, serde_json is:

serde_json-1.0.150-build-script-c6aeda35
serde_json-1.0.150-run-build-script-6a66cc2a
serde_json-1.0.150-lib-c4064345

The script is compiled, then run, then the library is compiled with what the script printed. rostnix makes a derivation of every unit, 54 of them for ripgrep. Trimmed down, the library looks like this in the generated code:

units."serde_json-1.0.150-lib-c4064345" = b.compile {
  name = "rustlib-serde_json-1.0.150";
  src = sources."serde_json-1.0.150";
  kind = "lib";
  rustcArgs = [ "--crate-type" "lib" "-C" "opt-level=3" "-C" "embed-bitcode=no" "-C" "debuginfo=1" /* ... */ ];
  deps = [ { name = "itoa"; unit = units."itoa-1.0.18-lib-8f4c7e2d"; } /* ... */ ];
  buildScript = units."serde_json-1.0.150-run-build-script-6a66cc2a";
  # ...
};

Cargo does not run at build time. The derivations run rustc and the build scripts themselves.

rustc Flags #

The unit graph says what to build and with which profile, but not with which flags. Cargo used to print its rustc command lines with --build-plan. The cargo 1.95 in nixpkgs 26.05 answers that with unexpected argument.

So rostnix works out the flags itself, by cargo’s rules. The debuginfo=1 above comes from ripgrep’s [profile.release]. LTO is the worst of it. Cargo decides per unit between -C lto, -C linker-plugin-lto, -C embed-bitcode=no and no flag at all, from the profile, the crate types and what depends on the unit.

To find out where I got them wrong, rostnix’s integration tests build small fixtures, rostnix itself and core-rs with cargo build -vv and cargo test -vv, and compare every rustc invocation, build script run and test run with what rostnix did for the same source. Apart from paths, hashes and a short list of known differences, the two have to match.

Crates #

gonixgo has to hash every module directory to find out where Nix would put it. Rust makes this easier, because the checksum in Cargo.lock is the SHA-256 of the .crate file:

[[package]]
name = "serde_json"
version = "1.0.150"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8014e44b4736ed0538adeecded0fce2a272f22dc9578a7eb6b2d9993c74cfb9"

That is the hash fetchurl wants, so the store path of every download follows from Cargo.lock and nothing has to be hashed:

sources."serde_json-1.0.150" = b.fetchCrate {
  pname = "serde_json";
  version = "1.0.150";
  sha256 = "e8014e44b4736ed0538adeecded0fce2a272f22dc9578a7eb6b2d9993c74cfb9";
  url = "https://static.crates.io/crates/serde_json/serde_json-1.0.150.crate";
};

Cargo downloads the crate while it plans. rostnix adds the file from cargo’s cache to the store under that path, and the fetch derivation finds its output already there. Only a machine that did not evaluate the project downloads anything at build time.

Git dependencies are fetched by builtins.fetchGit at the revision in Cargo.lock, with your own git credentials. Crates from other registries come out of cargo’s cache like the ones from crates.io, so private registries work too.

Source Files #

Each derivation should only get the source files it reads, or an edit anywhere rebuilds everything. In Go that is easy, go list names the files of every package. A Rust crate is whatever its root file pulls in with mod and include_str!, and a build script reads whatever it likes.

So each unit gets its package directory minus what it most likely does not read: the directories of other packages (ripgrep’s root package has all the others under crates/), tests/, examples/ and benches/ unless the unit builds one of those, and the root files of the package’s other binaries and tests. A file that the unit’s own root names with mod or include_str! stays. A unit that reads something outside its copy fails with “file not found”, and extraSrc in crateOverrides adds it back.

A build script run keeps the root files of the other targets, main.rs included, since there is no telling what it reads. And a file that nothing reads, the README say, still rebuilds its package when it changes. A src narrowed with lib.fileset avoids that.

Rebuild Times #

I built ripgrep 15.2.0 both ways on an M4 Pro, with buildRustPackage and cargoLock.lockFile on one side. Tests were off on both sides, since rostnix would otherwise run them:

The change was a comment added at the end of crates/core/main.rs. The numbers are medians, of five runs for the first build and ten for the rebuild, because the machine had other work running at the time.

After the change buildRustPackage compiles every crate again. rostnix does this:

these 3 derivations will be built:
  rustbsrun-ripgrep-15.2.0.drv
  rustbin-rg.drv
  ripgrep.drv

The build script runs again because it sees main.rs, which takes less than a tenth of a second. Then rg is compiled and the final package collects it. Compiling rg is three to five seconds of the 6.8, and most of the rest is evaluation, cargo’s planning included.

The first build is slower, as it was for Go. Part of that is max-jobs = 1, the Nix default, which builds one derivation at a time. With -j auto it takes 21 seconds, still more than buildRustPackage, because the longest chain of dependencies, from regex-syntax through regex-automata, globset and ignore to rg, is built one crate after another. Cargo overlaps those. It starts compiling a crate as soon as its dependencies have written their metadata, before their code is generated. rostnix gives rustc the finished libraries, so every crate in the chain waits for the one before it.

Tests #

gonixgo runs tests too now, it learned that after the post about it. rostnix runs them unless you set doCheck = false. It asks cargo a second time, for what cargo test would build, and every test executable becomes a derivation that compiles it and runs it. The final package depends on all of them, so a failing test fails the build.

For ripgrep that adds 11 units to the 54: the tests in rg’s own source, the integration tests, and serde_derive with the four crates it needs, which only the tests use. The other 54 are the same derivations as without tests. All 323 integration tests and 114 unit tests pass.

Editing one of the integration test files builds and runs the tests again, and nothing else:

these 3 derivations will be built:
  rusttest-rg.drv
  rusttest-integration.drv
  ripgrep.drv

rusttest-rg is in the list because a test may read anything in its package, so it sees tests/ too.

cargo-nix-plugin #

cargo-nix-plugin works out the plan itself, with a reimplementation of cargo’s resolver inside the plugin. It needs no cargo and no crate sources during evaluation, only the index entries of the crates, a few hundred bytes each. rostnix runs cargo, which downloads every crate the build needs before it answers, but the answer is cargo’s own.

It builds each crate in one derivation. The build script is compiled and run, then the library and the binaries are compiled. In rostnix those are separate units, so editing a binary does not rebuild the library next to it. buildRustCrate also has its own flags. In release mode every crate gets -C opt-level=3 and 16 codegen units, whatever [profile] says, so ripgrep’s debug = 1 would not be applied, and neither would LTO.

The tests of a workspace member are compiled in one derivation and run in another, runTests, which you add to your checks yourself. They are not part of the build, and the unit tests of binaries are not compiled. In rostnix every test executable is a unit, and the build depends on all of them.

cargo-nix-plugin does not need the unsafe flag, though.

The Unsafe Flag #

rostnix needs allow-unsafe-native-code-during-evaluation like gonixgo does. While it is on, any Nix expression you evaluate can run any program as you, so pass it on the command line for projects you trust. The rest of what I wrote about it there holds here too, direnv included.

Cargo runs during evaluation as you, with the download cache, registry settings and credentials of your cargo home. What you have set up for your own builds stays out. RUSTFLAGS and [build] in ~/.cargo/config.toml change nothing, the project’s own .cargo/config.toml does.

Limitations #

It is a day old. Doc tests and benches are not run, and path dependencies outside src and vendored sources are not supported. A build script that writes anywhere but OUT_DIR fails, because the package directory is read-only. The rustc flag rules follow cargo 1.95, and other toolchains are untested.

rostnix’s integration tests pass on an Apple Silicon Mac with Nix 2.26 and on x86_64 Linux with Nix 2.31. ripgrep and ruff 0.15.5, which has 451 units and some dependencies from git, have only been built on the Mac. Of cross builds, WebAssembly and Linux with musl are tested. aarch64 Linux built from x86_64 has been compiled once, but not run.

Alternatives #

If one derivation is fast enough for you, stay with buildRustPackage. If you mostly edit one crate and seldom touch Cargo.lock, crane gets you most of the way without the flag. crate2nix is the one to look at if you want a derivation per crate and a generated Cargo.nix does not bother you, and cargo-nix-plugin if you can load a plugin into every Nix that evaluates the project. rostnix goes one step finer and leaves the planning to cargo, and it needs the flag for that.

The code is at github.com/draganm/rostnix. If you try it, I would like to hear how it went.