Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixos/switch-to-configuration: Proper unit file parser and clean/fix lower part of the script #154809

Merged

Conversation

dasJ
Copy link
Member

@dasJ dasJ commented Jan 12, 2022

Motivation for this change

This replaces the naive K=V unit parser with a proper INI parser from a
library and adds proper support for override files. Also adds a bunch of
comments about parsing, I hope this makes it easier to understand and
maintain in the future.

There are multiple reasons to do so, the first one is just general
correctness with is nice imo. But to get to more serious reasons (I
didn't put in all that effort for nothing) is that this is the first
step torwards more clever restart/reload handling. By using a library
like Data::Compare a future PR could replace the current way of
fingerprinting units (which is to compare store paths) by comparing the
hashes. This is more precise because units won't get restarted because
the order of the options change, comments are added, some dependency of
writeText changes, .... Also this allows us to add a feature like
X-Reload-Triggers so the unit can either be reloaded when these change
or restarted when everything else changes, giving module authors the
ability to have their services reloaded without having to fear that
updates are not applied because the service doesn't get restarted.
Another reason why this feature is nice is that now that the unit files
are parsed correctly (and values are just extracted from one section),
potential future rewrites can just rely on some INI library without
having to implement their own weird parser that is compatible with this
script.

This also comes with a new subroutine to handle systemd booleans because
I thought the current way of handling it was just ugly. This also allows
overriding values this script reads in an override file.

Apart from making this script more compatible with the world around it,
this also fixes two issues I saw bugging exactly 0 (zero) people. First
is that this script now supports multiple override files, also ones that
are not called override.conf and the second one is that 1 and on are
treated as bools by systemd but were previously not parsed as such by
switch-to-configuration.

Related to #49528.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.05 Release Notes (or backporting 21.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@github-actions github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` labels Jan 12, 2022
@dasJ

This comment has been minimized.

@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 labels Jan 12, 2022
@dasJ dasJ force-pushed the feat/stc-proper-unit-file-parser branch from c6fcd61 to ce0cdde Compare January 19, 2022 11:14
@github-actions github-actions bot added 8.has: changelog 8.has: documentation This PR adds or changes documentation labels Jan 19, 2022
@dasJ
Copy link
Member Author

dasJ commented Jan 19, 2022

About me removing the comment from parseKeyValues: This is actually correct when parsing the output of systemctl show and not parsing a unit file.
This does not matter however because I also plan on dropping that subroutine entirely and replacing it by systemctl show --value --property so we don't have to do the splitting ourselves and can just let systemctl do it for us

I added a second commit that cleans up the lower part of the script and gets rid of parseKeyValues

This replaces the naive K=V unit parser with a proper INI parser from a
library and adds proper support for override files. Also adds a bunch of
comments about parsing, I hope this makes it easier to understand and
maintain in the future.

There are multiple reasons to do so, the first one is just general
correctness with is nice imo. But to get to more serious reasons (I
didn't put in all that effort for nothing) is that this is the first
step torwards more clever restart/reload handling. By using a library
like Data::Compare a future PR could replace the current way of
fingerprinting units (which is to compare store paths) by comparing the
hashes. This is more precise because units won't get restarted because
the order of the options change, comments are added, some dependency of
writeText changes, .... Also this allows us to add a feature like
`X-Reload-Triggers` so the unit can either be reloaded when these change
or restarted when everything else changes, giving module authors the
ability to have their services reloaded without having to fear that
updates are not applied because the service doesn't get restarted.
Another reason why this feature is nice is that now that the unit files
are parsed correctly (and values are just extracted from one section),
potential future rewrites can just rely on some INI library without
having to implement their own weird parser that is compatible with this
script.

This also comes with a new subroutine to handle systemd booleans because
I thought the current way of handling it was just ugly. This also allows
overriding values this script reads in an override file.

Apart from making this script more compatible with the world around it,
this also fixes two issues I saw bugging exactly 0 (zero) people. First
is that this script now supports multiple override files, also ones that
are not called override.conf and the second one is that `1` and `on` are
treated as bools by systemd but were previously not parsed as such by
switch-to-configuration.
@dasJ dasJ force-pushed the feat/stc-proper-unit-file-parser branch from ce0cdde to cbed020 Compare January 20, 2022 14:34
@dasJ dasJ changed the title nixos/switch-to-configuration: Proper unit file parser nixos/switch-to-configuration: Proper unit file parser and clean lower part of the script Jan 20, 2022
@dasJ dasJ force-pushed the feat/stc-proper-unit-file-parser branch 2 times, most recently from ec5fd2d to 30789fb Compare January 20, 2022 16:08
@dasJ
Copy link
Member Author

dasJ commented Jan 20, 2022

A classic while writing the test: I found that the current behaviour that detects whether a unit is in auto-restart was broken. This is now fixed and slightly refactored (second commit) and a proper test case is added.

- Fully get rid of `parseKeyValues` and use systemctl features for that
- Add some regex modifiers recommended by perlcritic
- Get rid of a postfix if
- Sort units when showing their status
- Clean the logic for showing what failed from `elif` to `next`
- Switch from `state` to `substate` for `auto-restart` because that's
  actually where the value is stored
- Show status of units with one single systemctl call and get rid of
  COLUMNS in favor of --full
- Add a test for failing units
@dasJ dasJ force-pushed the feat/stc-proper-unit-file-parser branch from 30789fb to b523726 Compare January 20, 2022 16:10
@dasJ dasJ changed the title nixos/switch-to-configuration: Proper unit file parser and clean lower part of the script nixos/switch-to-configuration: Proper unit file parser and clean/fix lower part of the script Jan 20, 2022
@dasJ
Copy link
Member Author

dasJ commented Jan 27, 2022

Any updates on this? I'm really hesitant to self-merge, especially with no reviews, but I really want to see things moving forward and to push out the next changes that I have prepared.

Copy link
Member

@stigtsp stigtsp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

  • Perl changes seem OK
  • Config::IniFiles is a stable and maintained package
  • Tests in switch-test.nix pass, and coverage seems good

(It would be nice if someone who knows this part of nixos better than me could have a closer look)

@lovesegfault lovesegfault merged commit 5f9b470 into NixOS:master Jan 27, 2022
@dasJ dasJ deleted the feat/stc-proper-unit-file-parser branch January 27, 2022 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: changelog 8.has: documentation This PR adds or changes documentation 8.has: module (update) This PR changes an existing module in `nixos/` 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants