nixos: doc: include all modules in manual generation

[oxij]

What? Why?

An edited version of #47177. See commit messages below.

git log

• nixos: doc: include all modules in manual generation

  Before this change man 5 configuration.nix would only show options of modules in
  the baseModules set, which consists only of the list of modules in
  nixos/modules/module-list.nix

  With this change applied all modules included in configuration.nix file will
  be used instead.

  This makes configurations with custom modules self-documenting. It also means
  that importing non-baseModules modules like gce.nix or azure.nix
  will make their documentation available in man 5 configuration.nix.

  This was originally implemented in nixos-manual: Include all modules in documentation generation #47177, edited for more configurability and
  rebased onto master by @oxij.

• nixos: doc: increase maxdepth for xsltproc

  See manual fails to build on master #37903 (comment)
  for details. With the previous patch and some custom modules included in
  configuration.nix the above bug is very easy to trigger.

  This is a simplest workaround I have. A proper solution would look like
  manual fails to build on master #37903 (comment).

nix-instantiate environment

• Host OS: Linux 4.9, SLNOS 19.03
• Nix: nix-env (Nix) 2.1.3
• Multi-user: yes
• Sandbox: yes
• NIXPKGS_CONFIG:

{
  checkMeta = true;
  doCheckByDefault = true;
}

nix-env -qaP diffs

• On x86_64-linux:

  • Updated (2):
    • tests.nixos-functions.nixos-test
    • tests.nixos-functions.nixosTest-test

• On aarch64-linux: ditto

• On x86_64-darwin: noop

/cc original author of #47177 @arianvp, documentation aficionados @grahamc @vcunat

[arianvp]

Thanks for moving this forward!

[danbst]

what is the rational here? AFAIK modules don't depend on their order.

[danbst]

This ignores pkgsModule module. Why ignore that?

[oxij]

- modules = modules ++ extraModules ++ baseModules ++ [ pkgsModule ]; + modules = baseModules ++ extraModules ++ [ pkgsModule ] ++ modules; what is the rational here? AFAIK `modules` don't depend on their order.

OCD. This way the order is the same everywhere. This order also makes sense semantically.

+ manualModules = baseModules ++ optionals cfg.nixos.includeAllModules (extraModules ++ modules); This ignores `pkgsModule` module. Why ignore that?

`pkgsModule` is an autogenerated thing defined at the top of `nixos/lib/eval-config.nix`, it has no options/documentation. When and if `nixpkgs.config` will get structured then this will need to be updated.

[danbst]

OCD

OK :)

I'd also like things like documentation only for custom modules, documentation only about used options, documentation about only changed options, but this is very out of scope in this context.

[infinisil]

Can you add a release note for this? I think this very much warrants one

[oxij]

Done.

[danbst]

I've tried building my system with this patch. Turns out, lots of my custom options are not documented, so I end up in:

error: Option `nixpkgs.latestPackages' has no description.
(use '--show-trace' to show detailed location information)

Fix was easy, add description = ""; to all options, but still it required fixes. Perhaps gradual enable is needed - first add option with false as default, then set it by default true in nixos-generate-config, then change default after a few releases.

[oxij]

Perhaps gradual enable is needed - first add option with `false` as default, then set it by default true in `nixos-generate-config`, then change default after a few releases.

Well, since adding descriptions to your custom options is a backwards-compatible change I think this breakage is of okay kind. But for convenience sake, I think, a better approach is to just change that "has no description" error into a warning. The new commit does that.

[infinisil]

Testing this on my setup uncovered that nixops' option descriptions apparently don't all evaluate:

error: attribute 'bootstrap' missing, at /nix/store/nsnycm9acirddfdwj0mwiqx88ilwp4mi-nixops-1.6.1pre2728_8ed39f9/share/nix/nixops/gce.nix:343:19

This should be fixed in nixops and nixpkgs' nixopsUnstable should be updated before this can be merged.

[danbst]

Change default to false (with respected release notes changes) and I'll merge. It can even be backported to 19.03.

Because this can fail configuration evaluation, we should warn users first. They update to 19.03, see some red warnings in configuration and know, that option declarations must be fixed before 19.09, because 19.09 will make this behavior default.

This is just an idea. Making default false will make life easier for maintainers.

[oxij]

I see, I agree that if this uncovers bugs in third-party things then we should wait with setting it to `true` by default. Rebased onto master, changed the default and documentation. This now adds release notes to 19.09. It's not like this is urgent or anything so enabling this by default in 20.03 should be fine, IMHO.

[danbst]

thanks @oxij and @arianvp !