nixos: add grsecurity module (RFC)

[thoughtpolice]

This module is a work-in-progress which refactors grsecurity support into a module, which can be easily enabled from a users configuration.nix.

There are a few reasons for doing this: the current way of enabling grsec support is incredibly un-nix like, as it requires choosing a kernel package and then appropriate options. This is error prone, and requires a lot of work for users when they really want an easy way to do it.

Second, grsec naturally encompasses some other things that can only be handled by a module, that are beyond the kernel package: in particular, mutable files must be installed appropriately into /etc/grsec (as gradm will use/update the policy files). There are also needed udev rules for /dev/grsec. Furthermore, some options like GRKERNSEC_CHROOT_CHMOD are extremely incompatible with Nix, and make it impossible to install software. So by default, the module disables this (although it works great if you deploy with NixOps).

Also, some options are incompatible in other ways: for example, GRKERNSEC_RANDSTRUCT breaks the virtualbox guest networking drivers (a bug I need to send to grsec upstream), and GRKERNSEC_LINK breaks on the 3.13.5 testing kernel (but not the 3.2 stable kernel). Also, if you disable sysctl support, things like systemd-sysctl must be disabled as they won't work.

Overall, having a module take care of this means there's a lot less pain. This module naturally takes control of boot.kernelPackages - I'm not sure how much precedent there is for that, but I think it's reasonable given the purpose.

This also moves the grsecurity overrides out of the kernel package and into the module itself, leaving only the patches in pkgs, which I think is probably where it belongs anyway.

Finally, as the grsecurity patches imply config.apparmor=true, the module also turns apparmor on as the patches are included (which is easily overlooked).

This is my first 'big' NixOS module that's not simply a service, so I'd appreciate input. If we can land this in some form, more exciting work can happen soon, including enforcement/management of RBAC policies and userspace hardening.

Here's an example of enabling the configuration:

  security.grsecurity.enable  = true;
  #security.grsecurity.stable = true; # enable stable 3.2.55 kernel
  security.grsecurity.testing = true; # enable testing 3.13.5 kernel
  security.grsecurity.config = {
    mode     = "auto";
    priority = "security";
    system   = "server";
    virtualisationConfig     = "host";
    hardwareVirtualisation   = true;
    virtualisationSoftware   = "kvm";

    # Extra protection options
    denyChrootChmod   = true;

    # Extra kernel options
    kernelExtraConfig = ''
      DEVKMEM? n
      HIBERNATION? n
      XEN? n
      HYPERVISOR_GUEST? n
      KVM_GUEST? n
      PARAVIRT? n
    '';
  };

I'm using this configuration with NixOps, and am successfully deploying Linux 3.13.5-grsec onto Hetzner with this module.

There are some things left TODO:

• Enable AppArmor patches by default
• Enable AppArmor module by default
• Automatically enable AppArmor configurations in modules by default.
  • So far, I think transmission is about the only one that does this.
  • We should review upstream apparmor configurations
• When first deployed with NixOps, systemd.services.systemd-sysctl.enable gets shadowed rather than cleanly disabled it appears, meaning I probably got this wrong somehow.
• Make sysctl options 'optional' (they will not appear in the systemd-sysctl config) if specified with null.
• Ensure kptr_restrict is null if grsecurity is enabled.
• Fix all option names
• Enable revision in module version.
  • Should probably be optional
  • Needs to override ./locaversion-grsec after patching
• Properly detect virtualbox guest drivers to disable RANDSTRUCT.
• Abstract out the building of grsecurity patches
• Enable vserver patches
• Fix kernel package configuration overrides
• Enable grsec-lock service for configurations that enable sysctl support
• Enable a gradm module for RBAC control. Ideally most of this could be declarative.
• Fix configuration of RANDSTRUCT and GRKERNSEC_LINKS
• Make common-config kernel options optional
  • When using grsec, you often want to cut down the amount of stuff built (and it makes testing easier), so several of these things should be optional (e.g. I have no need for bluetooth, NFC, or many of the extra filesystems).

/cc @wizeman

[wizeman]

First of all, again: great work!

Minor comment here: why have an option for RANDSTRUCT (and not the other grsec config options)?

I would rather let GRKERNSEC_CONFIG_AUTO determine whether to enable or disable RANDSTRUCT than let NixOS make the decision for me (unless there is a good reason).

And if some user wants to force-enable/force-disable this option he could just use kernelExtraConfig, right?

[thoughtpolice]

This is sort of on an as-needed basis, at the moment. Should there be coverage for every option? Ideally, that would be the case. But there are a lot of options.

Second, RANDSTRUCT is here because as I said above it breaks the vbox guest networking drivers, meaning it's impossible to do things like test grsec kernels in VirtualBox with NixOps. I have not yet discovered why this is the case (in particular other modules are compiled with the correct entropy string as a part of RANDSTRUCT). However, it does not break other modules, meaning RANDSTRUCT can be deployed successfully e.g. on Hetzner (where it works great).

I don't know how to predicate this on whether or not we're being deployed inside a VirtualBox guest - does anyone know how to do that? Then RANDSTRUCT can be an option, but optionally toggled if it might break something for the moment.

This also brings up a good point in that duplicated kernelExtraConfig options will be ignored (or possibly error) the kernel configuration script. So maybe we should make everything an actual option...

[wizeman]

Perhaps it's not the best way (and it's certainly not fool-proof), but in most cases we can find out if the system is going to be running as a VirtualBox guest if config.services.virtualbox.enable == true.

But if we're not running as a VirtualBox guest, I would rather not define RANDSTRUCT in the kernel config unless the user specified it...

[wizeman]

Bah, I'll just continue adding comments here otherwise this will get even more messy.

Just to clarify one of your comments, if I'm reading the generate-config.pl code correctly, duplicate kernel options are not ignored or throw an error. If you have, e.g.:

SOME_OPTION n
SOME_OPTION y

I think the option will be defined as "y", i.e., the later definitions override the earlier ones.

[wizeman]

BTW, this is why when you define these options in the module:

     DEVKMEM? n
     (...)
     XEN? n

... they don't get ignored or error out, even though they are already defined in pkgs/os-specific/linux/kernel/common-config.nix :-)

[wizeman]

Comments as of the current state:

• I'm not sure about the sysctl changes, I would like if somebody more experienced could take a look at them... (but I'm not sure who).
• In patches.nix, instead of duplicating the revision inside both the url and the revision attribute, it would be better to use ${revision} as part of the url string, so that they both always match. In fact, in your patch they are mismatched already... :-)
• I would still like if RANDSTRUCT would not be explicitly enabled or disabled by default in NixOS, i.e., I would prefer if we simply do not define it in the kernel configuration unless the user explicitly set a value (or if VirtualBox is enabled). We could just set the default to null, for example.
  I view this option as different from the other ones, because for the other ones that you defined, users should know relatively well what they should be set to (e.g. RBAC vs no RBAC), while for RANDSTRUCT it's not always clear what it should be set to (as it may have performance implications...). Also, since the implementation may change, it just seems safer to let upstream determine the default...

Apart from these issues, I would be OK with getting it merged, even if your TODO items are not all fixed yet.

[thoughtpolice]

Current status:

• gradm is now working-broken instead of broken-broken. Previously the default learning configuration would mark /nix/store as hidden, which would then cause gradm to get killed by ACL enforcement when it tries to load ld-linux-x86_64.so from /nix/store/...-glibc. However, it seems as if turning off the learning mode is now broken slightly.
• The grsec-lock service is in place and works well. I made a compromise: it is implied by multi-user after sysctl is run. This means it will always be run immediately upon boot - you can configure options using boot.kernel.sysctl, but enforcement of the new policies requires a reboot. This makes it relatively impossible to forget, but still lets you turn off things as you choose when you tweak your setup.

[wizeman]

👍 to your latest changes :)
It's looking really good, at least from a grsecurity perspective :) You seem to have fixed all the issues I mentioned and I also like your other improvements.

Of course, this is a large change and I'm also a bit unsure about the sysctl and kernel config changes, so we'll definitely have to get a more experienced person reviewing this as well.
When you think it's ready to merge let me know and we'll see if we can trick someone else into doing a review :)

[wizeman]

I just noticed: sha256 = sha256 can be better written as inherit sha256

[peti]

This looks really cool, guys! Thank you for doing this.

[shlevy]

Is there any way we can hook into <nixpkgs/pkgs/os-specific/linux/kernel/manual-config.nix> (which really needs to be renamed) to allow minimal kernel configuration and possibly more options besides 3.13 and 3.2?

[shlevy]

And is there any way we can have something akin to mutableUsers = false, where all of the traditionally mutable grsec configuration is actually managed by nix? Would be useful for deployments I expect.

[thoughtpolice]

Relaying some stuff I told Shea on IRC as answers to his questions:

1. I'd really like to hook this into the kernel configuration options and provide some means of having a 'minimal config'. My personal NixOps deployment has 70+ kernel modules that it disables - mostly things like lots of filesystems, and unused networking systems - that I deal with. It'd be nice to have a minimal kernel config that the grsec module could build on.

2. 3.2 and 3.13 are the only supported kernel versions, as they correspond to the grsecurity LTS/testing branches. Generally, stable follows the ubuntu LTS (so grsecurity stable will soon be 3.13 or possibly 3.14, after Trusty Tahir is released). Unstable follows the latest kernel, always - so it will become 3.14, 3.15, 3.16, etc..

   A related question: What about grsecurity updates on the NixOS stable branch? IMO, they should certainly go in, but the kernel release cycle vs the NixOS stable release cycle probably doesn't match, so testing will likely be out of date a lot, or we'll need to promote newer kernels to the stable branch so 'testing' grsecurity deployments don't break.

   Note that 'testing' just implies a newer kernel and possibly newer features/protections - I generally find it quite stable, but it DOES always track upstream closely.

3. Some thought needs to go into declarative gradm policies - for those who aren't aware, gradm works in a 'learning mode' where you let it watch your system and build a set of logs. This happens while your system is in production. Eventually, you compile the logs to a policy and enable it in the kernel, enforcing it.

   I'm not sure how to manage this off hand. It would be excellent if a service could automatically carry a policy however - the apparmor module already allows something like this. Likewise, Shea mentions it'd be good to at least allow you to specify a policy file - you could enable learning mode in your test environment, and ship the compiled policy to your production environment.

[shlevy]

Does this depend on the nixpkgs changes that were last-minute removed from @vcunat's stdenv branch to be useful? Or should it be merged as-is?

[thoughtpolice]

This doesn't depend on #1187 at all - all the needed packages are already upstream, although obviously it's a bummer that PR wasn't merged. I just need to rebase these changes and squash them up a bit to make them more coherent.

[shlevy]

OK. Let me know when you do, I don't think github pings when new commits are pushed to a PR.

[wizeman]

Ok, for now I have pushed the gradm commits, as they are simple fixes.
I can't do it right now, but I will take a look at the rest in the next few days :)

[wizeman]

I just took another look at your latest changes and I think it looks good 👍
But again, I don't know whether the sysctl and kernel changes are the way to go or not.

BTW, not sure what's up with the all the Haskell and the musl commits...

Regarding the comments above, here's what I think:

• +1 to being able to optionally select a minimal kernel config (but I wouldn't want that to block this PR from being merged).
• I agree with @thoughtpolice that kernel updates should go into NixOS's stable branch, at least for the default kernel (which looks like it will be 3.12) and for grsec's kernel branches ('stable' patch for sure, and I would say 'testing' too, at least for as long as it's feasible). Otherwise users will be running out-of-date kernels with security flaws.
• I think that ideally, NixOS should ship with good default gradm/RBAC policies for the packages it supports, adjusted to the user's configuration.nix options (e.g., similar to how the 'transmission' NixOS service has its own personalized AppArmor policy, but using a gradm/RBAC policy instead and expand it to more services/packages). But I agree with @thoughtpolice and @shlevy that until this is feasible, there should be a way to 'learn' the policy and later declaratively specify the learned policy in configuration.nix. However, I wouldn't expect the lack of this feature to block this PR from being merged, as I think this PR is already very useful as it is (i.e. without support for RBAC policies), and we can incrementally improve it over time to first add support for learning and setting a global RBAC policy, and later to make NixOS come with a good set of default RBAC policies for the services/packages it supports.

[thoughtpolice]

This is now merged in HEAD.

[thoughtpolice]

See #2213 for the follow up for kernel configuration options.