Documentation: pkgs.nixos

[olafklingt]

Problem

pkgs.nixos is missing documentaiton

Proposal

it should be documented in the nixpkgs manual but where?

Checklist

• checked latest Nixpkgs manual (source) and latest NixOS manual (source)
• checked open documentation issues for possible duplicates
• checked open documentation pull requests for possible solutions

Priorities

Add a 👍 reaction to issues you find important.

[roberth]

We might may want to deprecate this one in favor of a new one (TBD) in builders, analogous to testers.runNixOSTest.
That doesn't mean deprecated things shouldn't be documented though, so not a blocker.

[infinisil]

pkgs.nixos isn't just for testing though, it can be used to build a NixOS system for deployment, almost just like <nixpkgs/nixos>

[roberth]

Yeah, that's why I suggested builders instead, but that I've only hallucinated into "existence"...
It think it would make sense though, to move some builders there, although it'd be in competition with "builder sets"(?) such as dockerTools, writers and formats. The latter of which is not even clearly a builder set.

The poor little function already has a history of being pulled in multiple direction, making it a weird mishmash of interfaces.

1. Original idea: NixOS can be framed as a function from all the other options to the system.build option, which is a sort of return value. ✔️ implemented. More requirements follow.
2. It should be more general and return the values of all options ✔️ implemented
3. It should return something as general as nixos/eval-config. ✔️ implemented
4. It should be as general as nixos/eval-config. ❌ not implemented
5. It should not just be a function, but a facade for nixos/lib except with pkgs pre-applied ❓ good? bad?
6. It should be a builder ❓ how?

Being a builder would entail returning a derivation. A new one for NixOS could return system.build.toplevel, which is usually, but not always the root of the system's derivation DAG. It could then be extended to make all other options accessible, and the interface should separate modules(module?) and specialArgs, but not have any of the legacy arguments that merely forward to option-based counterparts, such as system (which is implied by the Nixpkgs context anyway). Furthermore, it should make the nixpkgs.* options read-only, like node.pkgs does in the NixOS VM test framework. This is what a Nixpkgs flavored NixOS builder would have to look like.
To add generality, we could add one or more passthru attributes to toplevel, either in toplevel itself, or in the builder. Ideally this is the returned configuration object (simplistically: nixosDrv.configuration._type == "configuration", and configuration.config.system.build.toplevel.configuration = configuration).
We could probably use lib.lazyDerivation to defer NixOS checks until instantiation (outPath, drvPath), making this entrypoint into NixOS just as efficient as the general one.

Going to back to the list of requirements, my conclusion is that pkgs.nixos is both weird and too limited. To document it, you'd have to take this essay, remove the suggestions and explain all the consequences and workarounds. Instead, we can have documentation like this, and avoid trapping people into code they'll probably have to migrate away from anyway.

Now concretely, what am I proposing?

(And apologies to anyone who finds this while it doesn't exist yet, or after the design has changed, although I find that unlikely, having spent way too much thought on these interfaces. Famous last words.)

---

pkgs.builders.nixos

Invoke NixOS with the current Nixpkgs as its pkgs, and return the buildable configuration as a package, plus all the options.

Type: { modules; specialArgs; } -> Package // { configuration }

Note: the nixpkgs.* options will not be writable by the configuration, instead forcing NixOS to use this Nixpkgs, including any overlays and other configuration.

Parameters

• modules: The configuration as a list of module system modules.
  Example: modules = [ ./configuration.nix ];
  Example:

  modules = [
    {
      fileSystems."/".device = "/dev/sda";
      boot.loader.grub.enable = false;
    }
  ];

• specialArgs: A set of module arguments, providing external context, which can not be modified by the modules, but can be used for imports.

Return value

The toplevel derivation, also known as config.system.build.toplevel. as well as its configuration attribute that allows any option to be read from the configuration.

Laws

configuration.config.system.build.toplevel.configuration = configuration

Note the use of =, meaning identical by construction - not just equal. (EDIT: maybe I should tone this down, as don't have this capability yet in the module system. See #260778 for loosely related thought, although that may defeat laziness compared to a configuration value aka evalModules return value)

---

Returning again to the list of requirements:

1. The system.build idea was flawed. Instead we double down on toplevel, which is usually the desired attribute from system.build anyway. The other attributes are still accessible through configuration. ✔️
2. pkg.configuration.config ✔️
3. pkg.configuration ✔️
4. Both modules and specialArgs ✔️
5. nixos/lib is unrelated. Such an interface facade can be done separately. ❌ but really just out of scope.
6. It behaves like a proper builder now.

And finally the requirements now satisfied, that I didn't number

• No more custom interface amalgamations, except for the configuration passthru, but that could be a general feature of toplevel anyway! => Easy to document => Easier to understand

Final thought: variations could be made to facilitate returning e.g. images, selecting different options to return as The Package. { modules; specialArgs; getPackage ? config: config.system.build.toplevel; } (and, implementation detail, outputs ? ["out"] because lazyDerivation needs to know, when it's not the usual toplevel).
This is like doing let it = builders.nixos args; in it.configuration.config.system.build.toplevel.vmImage // { inherit (it) configuration; }, but without the hassle; for those few who have an image based workflow.

[infinisil]

Generally this sounds pretty good to me!

Note: the nixpkgs.* options will not be writable by the configuration, instead forcing NixOS to use this Nixpkgs, including any overlays and other configuration.

Problem: A bunch of modules (both in Nixpkgs and third-party) depend on being able to modify nixpkgs.{overlays,config} options. I think those should not be read-only.

pkg.configuration.config ✔️

This is rather odd from an outsider perspective. The term "configuration" is already somewhat established for this, but I don't think it's too late to change that. Other ideas are eval[uation].config or just result.config.

We could probably use lib.lazyDerivation to defer NixOS checks until instantiation (outPath, drvPath), making this entrypoint into NixOS just as efficient as the general one.

Not a big fan of lazyDerivation, but it's probably the least bad option for now.

[roberth]

being able to modify nixpkgs.{overlays,config}

Correct. These users should actively avoid pkgs as part of an entrypoint into NixOS, to avoid confusion and generally for the Interface Segregation Principle.

pkg.configuration.config ✔️

This is rather odd from an outsider perspective.

It's the solution with least entropy in my perspective, but I'm open to other variations. Maybe what you're suggesting, or adding pkg.config as a shortcut.

Not a big fan of lazyDerivation, but it's probably the least bad option for now.

It's not a function to use widely, I agree, but this would be a prime use case for it.

[infinisil]

Correct. These users should actively avoid pkgs as part of an entrypoint into NixOS, to avoid confusion and generally for the Interface Segregation Principle.

It's a pretty bad experience to only become a user later though. And I don't think there needs to be a reason to make overlays/config read-only. At least overlays can be appended to an existing pkgs with .appendOverlays or .extends. We do need something similar for config though.

[roberth]

Maybe some sort of blueprint object, like what evalModules returns? Otherwise it's far too easy to do something needlessly expensive. As in call-by-need ;)
Actually maybe that's the logical conclusion of #231940 then, to always use that for Nixpkgs invocations.

Until we have that though, I think sometimes forbidding overlays is valid, and actually really good because it stops overlay interference, radically. Most things don't even need to be an overlay.

become a user later

Not sure actually what you mean here.

[infinisil]

become a user later

Not sure actually what you mean here.

Start out with pkgs.builders.nixos at first, then enable programs.ccache:

nixpkgs/nixos/modules/programs/ccache.nix

Lines 54 to 55 in b1cea2e

	nixpkgs.overlays = [
	(self: super: genAttrs cfg.packageNames (pn: super.${pn}.override { stdenv = builtins.trace "with ccache: ${pn}" self.ccacheStdenv; }))

And now you get an error that nixpkgs.overlays is read-only and are forced to stop using pkgs.builders.nixos.

[roberth]

The error should clarify that if you want NixOS to manage Nixpkgs, you need to use the normal NixOS entrypoint.

I agree that it's not super nice, but being nice is how you get chaos. Sometimes you have to draw a line. Boundaries are an important aspect of Nix as well. Some things that are forbidden, so that you get guarantees about something else.

If it's not a different approach to working with NixOS, then what's the point of having another entrypoint?

[roberth]

If I'm wrong and everyone hates it, we could allow it later, perhaps with a flag.
The NixOS test framework also has an opt out, in case you set node.pkgs for all your tests, but some tests need their own custom pkgs through the nixpkgs.* options.

[infinisil]

I guess this is a good reason to not have pkgs.nixos at all. It's recursive nature (Nixpkgs specifies NixOS specifies Nixpkgs) gives rise to such problems. How about deprecating the function and just delegating people to the actual NixOS entry points?

[roberth]

It's not recursive when NixOS doesn't specify Nixpkgs.
That's a different kind of NixOS that's safe from overlays' unintended side effects and faster to evaluate. Let's be very clear about that.
If they want overlays they can get it from the usual NixOS entrypoint, yes. We can also point that out in the error message.

[infinisil]

Imo nixpkgs.overlays should either be always supported (in which case you have to pay the performance penalty), or not supported at all (in which case you suffer modularity). I don't really care either way, but only supporting it sometimes sounds pretty bad.