ci: Set environment NIX_ABORT_ON_WARN=true

[lorenzleutgeb]

Resolves #155

See lib/trivial.nix.

Since this requires impure evaluation, and we want our flake to be pure in principle, split this into two separate evaluations, and also two jobs to hopefully save some time by parallelization. Caching the impure evaluation doesn't really make sense anyway.

[fricklerhandwerk]

I don't really understand the underlying problem. Is it that Nixpkgs/NixOS has impure expressions that fail hard on pure eval? How exactly does it help to split evaluations, and what is the partitioning?

[lorenzleutgeb]

I don't really understand the underlying problem.

There's no problem per se. It's just that @mightyiam and me would like to avoid stuff reaching main that generates warnings, i.e. calls lib.warn somehow, during evaluation.

Is it that Nixpkgs/NixOS has impure expressions that fail hard on pure eval?

In order to make lib.warn abort (like -Werror with gcc), you need to set the environment variable NIX_ABORT_ON_WARN. However, flakes evaluate purely by default, meaning that they don't have access to the network, information about time, the current machine, ... aaaand environment variables.

Therefore, we need to eval in impure mode to get the behavior we want, i.e. abort on warnings.

How exactly does it help to split evaluations, and what is the partitioning?

There's one impure evaluation, which will error if there's a warning, and one pure evaluation, which will not error if there's a warning. Since they cannot reuse any evaluation results (well, one is impure the other isn't...) it does not make sense to somehow run them sequentially and benefit from evaluation caching (if that would be supported anyway, see NixOS/nix#4279). Thus, run two jobs in parallel.

That way, the impure evaluation does not slow down nix flake check, and we still get a failed workflow if any of the two jobs fail.

Edit after both jobs have succeeded: As you can see, nix flake check takes 10x longer than nix flake check --no-build (obviously because it only evaluates and does not build anything). In the case that nix flake check has nothing to build (because everything is cached already), this difference might not be as pronounced, but I still see this as a confirmation that we don't add any "waiting for CI" time by adding the additional job.

[fricklerhandwerk]

I see, makes sense. Please add that sort of rationale into commit messages in the future (doesn't have to be that long).