NixOS: can't enable connman+wpa_supplicant due to missing /etc/wpa_supplicant.conf

[kwshi]

Describe the bug

When enabling connman with wpa_supplicant backend, nixos-rebuild switch fails to switch over to the new configuration. Specifically it tries to launch the wpa_supplicant service but crashes with the error message (full screenshot below):

awk: fatal: cannot open file `/etc/wpa_supplicant.conf' for reading: No such file or directory

Steps To Reproduce

Steps to reproduce the behavior:

1. From a fresh install of NixOS 22.11, set the following options in NixOS configuration services.connman={enable=true; wifi.backend="wpa_supplicant";}.
2. Run nixos-rebuild switch (assuming connman/wpa_supplicant aren't already active and running, which should be true on a fresh install, this should cause NixOS to try to start the wpa_supplicant service).
3. Crash!

Expected behavior

No crash; wpa_supplicant should launch without error, because, as discussed in previous instances of similar issues, connman interacts with wpa_supplicant directly and so shouldn't/doesn't need /etc/wpa_supplicant.conf to exist. If wpa_supplicant really depends on the existence of a configuration file, then the connman module should create a dummy blank configuration file instead of causing a crash.

Additional context

Here are some relevant past issues, though this one seems to be different than all of the other ones:

• Connman module does not work #199753 is relatively recent and might be the same issue as this one. Indeed, running nixos-rebuild boot instead of nixos-rebuild switch reproduces the same symptoms as that issue; the difference is that boot doesn't immediately try to switch over to the new configuration and so doesn't trigger the "failed to launch wpa_supplicant.service" error. I suspect that what's going on there is simply that systemd tries to start the wpa_supplicant service at boot, silently fails, and so upon startup connman fails to interact with wpa_supplicant and therefore reports "Not supported" when trying to scan.
• Wi-Fi fails to work when connman is enabled with networking.connman.enable = true. #23196 seems to be another instance of wpa_supplicant erroneously failing to start due to a missing /etc/wpa_supplicant.conf. However, that issue is almost six years old and exhibits a different error message than encountered here; the NixOS module source has evolved significantly since then, and so while the /etc/wpa_supplicant.conf aspect of the issue is the same, the underlying cause might not be. In fact, nixos/wireless: use udev to wait for interfaces #127595 claims to have resolved that issue over a year ago (on Aug 11, 2021), but the code introducing this error (as far as I can tell; details provided below) was added a week after that fix (52b9dd7 on Aug 17, 2021).

Detailed below is my attempt to track down the cause of the issue. The failed wpa_supplicant systemd service points to a wpa_supplicant-start script with the following contents:

#!/nix/store/wim4mqpn8lxhhr10p2kd070hyj152lil-bash-5.1-p16/bin/bash
set -e


# substitute environment variables
/nix/store/pndr3yb60iqx4igydv2irxi6s6y8rbvb-gawk-5.1.1/bin/awk '{
  for(varname in ENVIRON)
    gsub("@"varname"@", ENVIRON[varname])
  print
}' "/etc/wpa_supplicant.conf" > ""$RUNTIME_DIRECTORY"/wpa_supplicant.conf"

iface_args="-s -u -Dnl80211,wext -c "$RUNTIME_DIRECTORY"/wpa_supplicant.conf"

# detect interfaces automatically

# check if there are no wireless interfaces
if ! find -H /sys/class/net/* -name wireless | grep -q .; then
  # if so, wait until one appears
  echo "Waiting for wireless interfaces"
  grep -q '^ACTION=add' < <(stdbuf -oL -- udevadm monitor -s net/wlan -pu)
  # Note: the above line has been carefully written:
  # 1. The process substitution avoids udevadm hanging (after grep has quit)
  #    until it tries to write to the pipe again. Not even pipefail works here.
  # 2. stdbuf is needed because udevadm output is buffered by default and grep
  #    may hang until more udev events enter the pipe.
fi

# add any interface found to the daemon arguments
for name in $(find -H /sys/class/net/* -name wireless | cut -d/ -f 5); do
  echo "Adding interface $name"
  args+="${args:+ -N} -i$name $iface_args"
done


# finally start daemon
exec wpa_supplicant $args

In particular the crash is caused by the awk invocation (under the substitute environment variables comment): it tries to read from /etc/wpa_supplicant.conf but fails because that file doesn't exist. Grepping for these lines in the NixOS module source code (currently the nixos-22.11 branch as of writing) takes us to this code in nixos/modules/services/networking/wpa_supplicant.conf:

nixpkgs/nixos/modules/services/networking/wpa_supplicant.nix

Lines 124 to 128 in cdead16

	${pkgs.gawk}/bin/awk '{
	for(varname in ENVIRON)
	gsub("@"varname"@", ENVIRON[varname])
	print
	}' "${configFile}" > "${finalConfig}"

Viewing git blame of this code points to commit 52b9dd7 by @rnhmjoj.

I don't fully understand what the aforementioned code is meant to do; nor do I know exactly how connman/wpa_supplicant operates, so I'm not sure what solution to offer here, though I'm guessing the fix is relatively simple--omit this code when the configuration file isn't needed (e.g., when wpa_supplicant is enabled through connman). Alternatively (as was previously suggested in #23196) have the connman module auto-create a blank dummy /etc/wpa_supplicant.conf (see the workaround suggested by #23196 (comment)). I'm sure more knowledgeable people than me have well-formed thoughts on this.

Workaround

Until the issue is fixed officially, the issue can be locally sidestepped either by manually creating a dummy conf file (sudo touch /etc/wpa_supplicant.conf), or, if you'd really rather do things declaratively, with the following configuration snippet suggested by #23196 (comment):

environment.etc."wpa_supplicant.conf".text = lib.mkIf
    config.services.connman.enable
    ''
      # Dummy config file. Connman uses wpa_supplicant directly.
    '';

Notify maintainers

@rnhmjoj

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

• system: "x86_64-linux"
• host os: Linux 5.15.86, NixOS, 22.11 (Raccoon), 22.11.20230112.6a3f999
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.11.1
• channels(kshi): ""
• channels(root): ""
• nixpkgs: /nix/store/mi6xqy0h44p5ryh4sh8p8v3rwqgar0hb-source

[rnhmjoj]

Thank you for the detailed report.

I think I've never actually solved the issue with wpa_supplicant not starting without a configuration file: when I closed #23196 I meant I had solved the race condition with the daemon and networking card discovery.

The problem is that the networking.wireless module expects either /etc/wpa_supplicant.conf to exist or something to be configured declarative, but connman does neither: it uses dbus to control the daemon.

Unfortunately it seems the connman module is unnmantained, so no one ever fixed it.
I'll try to fix this now and write a simple test so that future changes to networking.wireless won't outright break it.

[AndersonTorres]

Do this help the issue #199753?

P.S.: I would like to maintain connman.

[rnhmjoj]

Yes, #212685 fixed this issue, but I haven't backported it to 22.11.
It shouldn't be a problem, though, It's just that I didn't have time to do it.