Building a NixOS system with the linux_rpi* kernels fails with Module ... not found in directory

[pstn]

Describe the bug

Building a sd-image for the raspberry-pi 4 fails with
modprobe: FATAL: Module sun4i-drm not found in directory /nix/store/azccsmgqmvhy8vmdr1f4rcpf1rbxvgx5-linux-5.10.52-1.20210805-modules/lib/modules/5.10.52

Steps To Reproduce

Steps to reproduce the behavior:
run nix-build '<nixpkgs/nixos>' -A config.system.build.sdImage -I nixos-config=./minimal.nix --argstr system aarch64-linux

with minimal.nix being:

{ config, pkgs, lib, ... }:
{

 imports = [
   <nixpkgs/nixos/modules/installer/sd-card/sd-image-aarch64-installer.nix>
 ];
  boot = {
    kernelPackages = pkgs.linuxPackages_rpi4;
  };
}

Expected behavior

An image is being produced

Additional context

Seems like it is this issue

Notify maintainers

@samueldr

Metadata

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
- system: `"x86_64-linux"`
- host os: `Linux 5.15.11, NixOS, 22.05 (Quokka)`
- multi-user?: `yes`
- sandbox: `yes`
- version: `nix-env (Nix) 2.5.1`
- channels(philipp): `""`
- channels(root): `"home-manager, nixos, nixos-hardware"`
- nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[samueldr]

Caused by:

• nixos/stage-1: Do not allow missing kernel modules in initrd #78430

See:

• Use conditionals for all architecture dependent initrd modules #109280
• Cross-compiling SD card image for Rasperry Pi (armv6l) on nixos-unstable branch fails #126755

Workaround:

• Cross-compiling SD card image for Rasperry Pi (armv6l) on nixos-unstable branch fails #126755 (comment)

The linux_rpi4 kernel has a lot of modules not built, and the way the kernel modules are configured to be included was changed, breaking the non-default use case most of the time.

[samueldr]

(I took the liberty of renaming the issue to better describe the underlying problem, since I'm likely to refer to it in the future)

[Mic92]

This module should be not included than for rpi4 kernel:

nixpkgs/nixos/modules/profiles/all-hardware.nix

Line 68 in 996536a

"sun4i-drm"

[samueldr]

Not in a reliable manner, and it's not a raspberry pi issue, but a kernel config difference issue.

Even mainline kernels may face such issues. The simpledrm changes we should do to the config will exhibit the same issue, but across different kernel revisions. (Though yeah, the package version can be used to modulate.)

Furthermore, I believe this is going to be a game of whack-a-mole to play on every module addition.

The semantics about kernel modules in stage-1 were changed in a way that makes them more correct (can't be missing), but this was done without an escape hatch.

The more correct solution, I think, would be to move away from lists, which we can't edit, and use an attrset with values indicating "mandatory", "optional", and a value to "remove" an entry (e.g. null). But this would need to be done in a way that is not disruptive to current setups, and that's where I lose steam.

[eyJhb]

Unsure if this is the same issue, but I am getting this on rpi3 with ahci

building '/nix/store/najsyhz7l9zgb9rwykfpl4ppaa5x9bwv-linux-5.10.92-1.20220118-modules-shrunk.drv'...
kernel version is 5.10.92
root module: md_mod
  copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/drivers/md/md-mod.ko.xz
root module: raid0
  dependency already copied: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/drivers/md/md-mod.ko.xz
  copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/drivers/md/raid0.ko.xz
root module: raid1
  dependency already copied: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/drivers/md/md-mod.ko.xz
  copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/drivers/md/raid1.ko.xz
root module: raid10
  dependency already copied: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/drivers/md/md-mod.ko.xz
  copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/drivers/md/raid10.ko.xz
root module: raid456
  copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/lib/raid6/raid6_pq.ko.xz
  copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/arch/arm64/lib/xor-neon.ko.xz
  copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/crypto/xor.ko.xz
  dependency already copied: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/drivers/md/md-mod.ko.xz
  copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/crypto/async_tx/async_tx.ko.xz
  copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/crypto/async_tx/async_xor.ko.xz
  copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/crypto/async_tx/async_pq.ko.xz
  copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/crypto/async_tx/async_memcpy.ko.xz
  copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/crypto/async_tx/async_raid6_recov.ko.xz
  copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/drivers/md/raid456.ko.xz
root module: ext2
  copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/fs/ext2/ext2.ko.xz
root module: ext4
  builtin dependency: ext4
root module: ahci
modprobe: FATAL: Module ahci not found in directory /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92
error: builder for '/nix/store/najsyhz7l9zgb9rwykfpl4ppaa5x9bwv-linux-5.10.92-1.20220118-modules-shrunk.drv' failed with exit code 1;
       last 10 log lines:
       >   copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/crypto/async_tx/async_pq.ko.xz
       >   copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/crypto/async_tx/async_memcpy.ko.xz
       >   copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/crypto/async_tx/async_raid6_recov.ko.xz
       >   copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/drivers/md/raid456.ko.xz
       > root module: ext2
       >   copying dependency: /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92/kernel/fs/ext2/ext2.ko.xz
       > root module: ext4
       >   builtin dependency: ext4
       > root module: ahci
       > modprobe: FATAL: Module ahci not found in directory /nix/store/9fy5l09vxxvjrz5dxvdb4mg3lspjadsn-linux-5.10.92-1.20220118-modules/lib/modules/5.10.92
       For full logs, run 'nix log /nix/store/najsyhz7l9zgb9rwykfpl4ppaa5x9bwv-linux-5.10.92-1.20220118-modules-shrunk.drv'.
error: 1 dependencies of derivation '/nix/store/[1gxg911gidg49md5n8vll8gfdzbqw254-stage-1-init.sh](http://1gxg911gidg49md5n8vll8gfdzbqw254-stage-1-init.sh/).drv' failed to build
error: 1 dependencies of derivation '/nix/store/a404g26v1gnq8bg7p2hab09askcbv7da-initrd-linux-5.10.92-1.20220118.drv' failed to build
error: 1 dependencies of derivation '/nix/store/wiig4rhimxigcf5gbfcrpzd5gfabpch7-nixos-system-hephaestus-22.05pre-git.drv' failed to build
error: 1 dependencies of derivation '/nix/store/qh6c45c035j7wph07qjjacmz4k98q5qs-deploy-hephaestus.drv' failed to build
error: 1 dependencies of derivation '/nix/store/4hiabk2fy4idgvk85icjb7k6hxwxm6wx-deploy.drv' failed to build

[kradalby]

I am running into this issue as well and as much as I dont like people posting "this happens to me too", I have a maybe stupid cluster:

How does the RPi image get built centrally? I cannot seem to figure out how I can work around this for now, but I'm puzzled by how this works some places?

[tobiasBora]

@kradalby what do you mean? For now the workaround is, as explained above, to add in the config:

overlays = [
  (final: super: {
    makeModulesClosure = x:
      super.makeModulesClosure (x // { allowMissing = true; });
  })
];

Source: #126755 (comment)
Edit: sorry for the mess-up with closing the ticket, my phone went crazy.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixos-raspberry-pi-no-hdmi-during-boot/23100/1

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rpi4-cant-usb-boot-from-btrfs-with-rpi-kernel/28668/1

[siraben]

Ran into this as well when trying to build the following flake with nix build .#images.rpi2

{
  description = "Build RPI image";
  inputs.nixpkgs.url = "github:nixos/nixpkgs/nixos-23.11";
  outputs = { self, nixpkgs }: rec {
    nixosConfigurations.rpi2 = nixpkgs.lib.nixosSystem {
      modules = [
        "${nixpkgs}/nixos/modules/installer/sd-card/sd-image-raspberrypi.nix"
        {
          nixpkgs.config.allowUnsupportedSystem = true;
          nixpkgs.hostPlatform.system = "armv7l-linux";
          nixpkgs.buildPlatform.system = "x86_64-linux";
        }
      ];
    };
    images.rpi2 = nixosConfigurations.rpi2.config.system.build.sdImage;
  };
}

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/does-pkgs-linuxpackages-rpi3-build-all-required-kernel-modules/42509/2

[Svenum]

Same issue here by building rpi4 kernel

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/fdt-error-notfound-on-device-tree-overlays/54911/2

[gcoremans]

I'm running into a different error when attempting to crosscompile the linux_rpi2 kernel from x86_64 to armv7l:

  CC [M]  drivers/iio/adc/vf610_adc.o
  CC [M]  drivers/iio/adc/viperboard_adc.o
  CC [M]  drivers/iio/adc/xilinx-xadc-core.o
  CC [M]  drivers/iio/adc/xilinx-xadc-events.o
  CC [M]  drivers/iio/adc/sd_adc_modulator.o
  LD [M]  drivers/iio/industrialio.o
  LD [M]  drivers/iio/pressure/bmp280.o
  LD [M]  drivers/iio/pressure/st_pressure.o
  LD [M]  drivers/iio/adc/xilinx-xadc.o
  AR      built-in.a
  AR      vmlinux.a
  LD      vmlinux.o
  OBJCOPY modules.builtin.modinfo
  GEN     modules.builtin
  GEN     .vmlinux.objs
  MODPOST Module.symvers
ERROR: modpost: "__aeabi_uldivmod" [drivers/pwm/pwm-rp1.ko] undefined!
ERROR: modpost: "__aeabi_uldivmod" [drivers/media/platform/raspberrypi/rp1_cfe/rp1-cfe.ko] undefined!
ERROR: modpost: "__aeabi_ldivmod" [drivers/media/platform/raspberrypi/rp1_cfe/rp1-cfe.ko] undefined!
make[1]: *** [../scripts/Makefile.modpost:126: Module.symvers] Error 1
make: *** [../Makefile:1966: modpost] Error 2

I do not know if this is the same issue or a different bug affecting the linux_rpi2 kernel, as the suggested overlay posted here does not resolve it.

[LongHairedHacker]

I'd say the __aeabi_uldivmod is a different issue. I ran into the same issue when trying to compile the kernel for the Pi 1 (armv6l-linux) using the nixos-24.11 branch.
Problem here seems to be that the kernel module is trying to do a 64bit division on a 32bit cpu. That has to be done in software, but the linker can't find the function/symbol actually implementing it.

It's also not necessarily new.
https://discourse.nixos.org/t/nixos-on-raspberry-pi-zero-w/38018/28
Somebody had the same problem back in march.
Also I encountered the same thing with my Pi1 on 24.05. Never figured out a fix,
replaced most of the hardware I had with newer Pis.