Generate API docs with Doxygen

[Ericson2314]

Motivation

The motivation is as stated in issue #7814: even though the the C++ API is internal and unstable, people still want it to be well documented for sake of learning, code review, and other purposes that aren't predicated on it being stable.

Per the brief discussion with the Nix team it is:

1. Not built by default as part of the main derivation
2. Can be built in the dev shell
3. Will be built by CI

Basically, it is done analogous to the coverage CI job, which is similarly a "bonus feature".

Context

Fixes #7814

Many things were modeled off of #7901. Thanks @roberth!

Checklist for maintainers

Maintainers: tick if completed or explain if not relevant

• agreed on idea
• agreed on implementation strategy
• tests, as appropriate
  • functional tests - tests/**.sh
  • unit tests - src/*/tests
  • integration tests - tests/nixos/*
• documentation in the manual
• code and comments are self-explanatory
• commit message explains why the change was made
• new feature or incompatible change: updated release notes

[abathur]

This FWIW probably isn't relevant for the use case here, but since nix already leans on sqlite I figure I should mention that doxygen has an optional/experimental sqlite output :)

[Ericson2314]

If that is useful for e.g. interactive IDE features, it might indeed be relevant. We are, after all, targeting people that want to work on Nix itself, since this is not part of Nix's public interface.

[abathur]

I haven't really looked into doxygen-output-consuming ide tooling.

It could, but I suspect if it already exists it's probably built on the xml output (it's existed longer, it's default/~stable).

But it does make it a little easier to build some kinds of tooling.

[roberth]

A devdoc output would be nice, but an independent derivation should be even nicer, although I don't think we can really pull that off yet without overcomplicating things.

[Ericson2314]

Yeah I would do that after your --enable-tests PR, @roberth :)

[fricklerhandwerk]

Triaged in the Nix team meeting:

We absolutely want this, but don't include it in the manual to avoid making the impression this is somehow a public interface. It's just to improve developer experience.

Decision: Please add a it as a new makefile target and separate derivation output.

Complete discussion

• @tomberek: inspecting structural changes visually will be much easier than reading header files
• @Ericson2314: there will also be more information for the reviewer
• @fricklerhandwerk: we already agreed we want this
  • @tomberek: the open issue was how to make unmistakably clear that this is not a public interface
    • could simply add a different make target or derivation output, bt not include it in the manual
• @Ericson2314: a lot of comments will have to be updated to follow the doxygen convention, this may be a good beginner project
  • @edolstra: wouldn't want beginners document code they don't understand
    • the only thing worse than missing documentation is incorrect documentation
  • @fricklerhandwerk: disagree on the premise; had multiple pairing and mob sessions digging through the code to understand what it does, and it produced sensible documentation
    • it helps a lot with onboarding beginners and spurs a discussion with maintainers
  • @edolstra: PRs are fine, but they should absolutely not be merged without review by someone knowledgeable with the code
    • the documentation should be authoritatively correct
    • @fricklerhandwerk: yes, obviously

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-03-03-nix-team-meeting-minutes-37/25998/1

[Ericson2314]

OK I think this is ready now!

[fricklerhandwerk]

Example class diagram rendered:

[fricklerhandwerk]

Discussed in Nix team meeting:

• @edolstra: we want API docs, but not sure about doxygen specifically (it's atrociously ugly)
  • can go for it if no better alternatives available, don't know any
    • (no one knows either)
    • @roberth: https://hdoc.io
  • the real issue is that our source code is not suitable to provide useful doxygen output
    • needs nice class hierarchies, and we don't have that
    • everything ends up in the giant nix namespace, and that's not useful organisation
    • may incenetivise refactorings though in order to get closer to that, e.g. of the util functions
• @Ericson2314: the subject came up related to the parts with more inheritance structure, in order to provide better overview visually
  • it's already delivering value there
• @edolstra: why is the doxygen config 2700 LOC?
  • @Ericson2314: took the defaults
  • @edolstra: try to avoid specifying unmodified fields, only provide overrides
  • @Ericson2314: will do that if possible
• @edolstra: should add a Hydra build result so the docs can be browsed from Hydra
  • @fricklerhandwerk: could link it from the hacking guide
  • @edolstra: Hydra can redirect to the latest build
  • @Ericson2314: this is downstream of that PR though
• Sphinx has some support, but it turns out it uses Doxygen too!
  • @Ericson2314: Seems like a good follow-up task then.
• agreement: idea approved, fix the config file

[roberth]

• needs nice class hierarchies, and we don't have that

We do. These docs are really helpful for getting an overview of how CLI commands are composed through mix-ins.

[roberth]

Let's make clear that it's our internal api.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-03-06-nix-team-meeting-minutes-38/26056/1

[Ericson2314]

OK @edolstra, I think I fixed everything brought up in the meeting. It's all yours now!

[Ericson2314]

Oops I forgot to note hydra build products as discussed in the meeting. This is done now.

[roberth]

Looks like a good start.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/this-month-in-nix-docs-1-march-2023/26913/1