ANSSI high profile triggers OpenSCAP error

[jan-cerny]

Description of problem:

The upstream CI job testing-farm:centos-stream-8-x86_64 and testing-farm:centos-stream-9-x86_64 fail on the test /Sanity/machine-hardening/anssi_bp28_high. There is an unwanted error message E: oscap: Can't statvfs /proc/sys/fs/binfmt_misc: errno=19, No such device. appearing in the final scan after reboot. This message appears during evaluation of rule audit_rules_privileged_command. This message comes from oscap from partition probe. In the rule audit_rules_privileged_command we use partition test for all paths.

My theory is that this is caused by remediation of rule kernel_config_binfmt_misc. EDIT: the rule kernel_config_binfmt_misc has no remediation.

SCAP Security Guide Version:

current upstream master as of 2023-04-11 as of head d8d1052

Operating System Version:

CS 8 , CS 9

Steps to Reproduce:

1. scan, remediate, reboot and then scan again with the --profile xccdf_org.ssgproject.content_profile_anssi_bp28_high

Actual Results:

E: oscap: Can't statvfs /proc/sys/fs/binfmt_misc: errno=19, No such device.

Expected Results:

no error

Additional Information/Debugging Steps:

can be related to big ANSSI update in #10334

[marcusburghardt]

The #10334 PR included the kernel_config_binfmt_misc rule which was also my initial guess. But this templated rule has no remediation, only the OVAL is available in kernel_build_config template.

[ggbecker]

This is the OVAL check for kernel_config_binfmt_misc in RHEL9

    <def-group>
      <definition class="compliance" id="kernel_config_binfmt_misc"
      version="1">
        <metadata>
            <title>Disable kernel support for MISC binaries</title>
                <affected family="unix">
                    <platform>multi_platform_all</platform>
                </affected>
            <description>The kernel CONFIG_BINFMT_MISC should have value n</description>
        </metadata>
        <criteria operator="OR">
          <criteria operator="AND">
            <criterion comment="Check presence of build configuration of installed kernels"
            test_ref="test_kernel_config_binfmt_misc" />
            <criterion comment="Ensure all kernels have the config"
            test_ref="test_all_kernels_config_binfmt_misc_compliant" />
          </criteria>
     
          <criterion comment="Check absense of build configuration of installed kernels"
          test_ref="test_kernel_config_binfmt_misc_absence" />
     
        </criteria>
      </definition>
     
      <ind:textfilecontent54_test check="all"
      comment="Check /boot/config-.* files for CONFIG_BINFMT_MISC=n"
      id="test_kernel_config_binfmt_misc" version="1">
        <ind:object object_ref="object_kernel_config_binfmt_misc" />
        <ind:state state_ref="state_kernel_config_binfmt_misc" />
      </ind:textfilecontent54_test>
     
      <ind:textfilecontent54_object id="object_kernel_config_binfmt_misc" version="1">
        <ind:filepath operation="pattern match">^/boot/config-.*$</ind:filepath>
        <ind:pattern operation="pattern match">^CONFIG_BINFMT_MISC="?(.*?)"?$</ind:pattern>
        <ind:instance datatype="int" operation="greater than or equal">1</ind:instance>
      </ind:textfilecontent54_object>
      <ind:textfilecontent54_state id="state_kernel_config_binfmt_misc" version="1">
        <ind:subexpression operation="equals" datatype="string">n</ind:subexpression>
      </ind:textfilecontent54_state>
     
     
     
      <ind:textfilecontent54_test check="all" check_existence="none_exist"
      comment="Check /boot/config-.* files for absence of CONFIG_BINFMT_MISC"
      id="test_kernel_config_binfmt_misc_absence" version="1">
        <ind:object object_ref="object_kernel_config_binfmt_misc" />
      </ind:textfilecontent54_test>
     
     
      <ind:variable_test check="all" check_existence="all_exist"
      comment="Check if all installed kernels are compliant"
      id="test_all_kernels_config_binfmt_misc_compliant" version="1">
        <ind:object object_ref="object_var_kernel_config_binfmt_misc_count" />
        <ind:state state_ref="state_var_kernel_config_binfmt_misc" />
      </ind:variable_test>
     
      <ind:variable_object id="object_var_kernel_config_binfmt_misc_count" version="1">
        <ind:var_ref>local_var_config_binfmt_misc_count_kernels_installed</ind:var_ref>
      </ind:variable_object>
     
      <ind:variable_state id="state_var_kernel_config_binfmt_misc" version="1">
        <ind:value operation="equals" datatype="int"
        var_ref="local_var_config_binfmt_misc_count_compliant_configs" />
      </ind:variable_state>
     
      <local_variable comment="Count number of kernels installed" datatype="int"
      id="local_var_config_binfmt_misc_count_kernels_installed" version="1">
        <count>
          <unique>
            <object_component object_ref="object_kernel_config_binfmt_misc_files"
            item_field="filepath" />
          </unique>
        </count>
      </local_variable>
     
      <local_variable comment="Count number of configs found" datatype="int"
      id="local_var_config_binfmt_misc_count_compliant_configs" version="1">
        <count>
          <unique>
            <object_component object_ref="object_kernel_config_binfmt_misc"
            item_field="filepath" />
          </unique>
        </count>
      </local_variable>
     
      <unix:file_object comment="Collect the kernel config files" id="object_kernel_config_binfmt_misc_files" version="1">
          <unix:path>/boot</unix:path>
          <unix:filename operation="pattern match">^config-.*$</unix:filename>
      </unix:file_object>
    </def-group>

[jan-cerny]

@ggbecker Should we see something specific there?

Reminder that the error happens during evaluation of rule audit_rules_privileged_command.

[ggbecker]

@ggbecker Should we see something specific there?

Reminder that the error happens during evaluation of rule audit_rules_privileged_command.

Probably not, it's just that we were wondering if it would be related to the rule kernel_config_binfmt_misc, but looking at the OVAL there is nothing that would justify the problem.

[jan-cerny]

Yes, as Marcus mentioned the rule kernel_config_binfmt_misc is probably unrelated because the template used in the rule doesn't provide any remediation.

[mildas]

@jan-cerny But the error comes from openscap scan so the problem might be related to OVAL test.

[jan-cerny]

@mildas yes, it's related to an OVAL partition_test that is a part of rule audit_rules_privileged_commands

[jan-cerny]

@marcusburghardt In that test we are scanning mountpoints matching ^/.*$. I would like to know if we could exclude /proc from that?

[marcusburghardt]

The relevant OVAL from the audit_rules_privileged_commands rule is this:

  <!-- First define OVAL entities that can be reused across tests below -->
  <linux:partition_state id="state_audit_rules_privileged_commands_dev_partitons" version="1">
    <linux:device operation="pattern match">^/dev/.*$</linux:device>
  </linux:partition_state>

  <linux:partition_state id="state_audit_rules_privileged_commands_nosuid_partitons" version="1">
    <linux:mount_options datatype="string" entity_check="at least one"
      operation="equals">nosuid</linux:mount_options>
  </linux:partition_state>

<linux:partition_state id="state_audit_rules_privileged_commands_noexec_partitons" version="1">
    <linux:mount_options datatype="string" entity_check="at least one"
      operation="equals">noexec</linux:mount_options>
  </linux:partition_state>

  <!-- This object is created mainly to improve performance when collecting file objects.
       Here all mount points are collected and filtered to include only devices under /dev in
       order to ignore special file systems. Then, the mount options are checked to exclude
       file systems mounted with nosuid or noexec. The privileged commands can't execute on these
       file systems, so there is no reason to proble these file systems. -->
  <linux:partition_object id="object_audit_rules_privileged_commands_exec_partitions" version="1">
    <linux:mount_point operation="pattern match">^/.*$</linux:mount_point>
    <filter action="include">state_audit_rules_privileged_commands_dev_partitons</filter>
    <filter action="exclude">state_audit_rules_privileged_commands_nosuid_partitons</filter>
    <filter action="exclude">state_audit_rules_privileged_commands_noexec_partitons</filter>
  </linux:partition_object>

Based on the error message, I guess it is generated when processing this partition_state:

  <linux:partition_state id="state_audit_rules_privileged_commands_dev_partitons" version="1">
    <linux:device operation="pattern match">^/dev/.*$</linux:device>
  </linux:partition_state>

It seems the proc is detected as a partition during the scan but for some reason there is no device attribute when checking <linux:device operation="pattern match">^/dev/.*$</linux:device>. At this point the error should be generated, if my assumption is correct.

If so, the idea from @jan-cerny to adjust the <linux:mount_point operation="pattern match">^/.*$</linux:mount_point> regex to exclude /proc will likely work for this specific error. Even without this error, if the /proc was indeed a valid partition detected by this partition object, it would be excluded by the filters. Therefore, it is safe to remove the /proc partition via regex.

On the other hand, I am not sure why the /proc was detected as a partition and consequently why it causes an error regarding the device while other pseudo file systems don't. Maybe other similar cases can appear in the future and a proper solution would be around this partition_state object in the scanner level. @jan-cerny , maybe you have more information about this object on the scanner side? What is the value for the device attribute related to the /proc mount point`?

In any case, manually excluding the /proc via regex sounds like a feasible workaround in short-term.

[jan-cerny]

@marcusburghardt

This OVAL partition_test should match all mount points on the target system. Here is what OpenSCAP partition_probe does when evaluates this OVAL test: It iterates over all entries in /proc/mounts. It uses the standard function getmntent_r (see man 3 getmntent_r for more details). For each entry this function gives us struct mntent and we process the mntent->mnt_dir which contains a "file system path prefix" further. We match the regex on it. The regex matches all mountpoints. In other words, we process all mount point paths that you can see in /proc/mounts. Then, we call statvfs() and when this fails this shows the error message. Take a look to src/OVAL/probes/unix/linux/partition_probe.c around line 171.

On the other hand, I am not sure why the /proc was detected as a partition

On my local laptop I can see the problematic path /proc/sys/fs/binfmt_misc has a special entry in my /proc/mounts, and as I mentioned OpenSCAP iterates over all entries in /proc/mounts:

systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21880 0 0

Maybe other similar cases can appear in the future

Possibly, we might need special handling for the "pseudofilesystems" in oscap.

[vojtapolasek]

Hello. So I also did some investigation. It seems that this file system is handled by Systemd. I am attaching recording of a session where I show it.
I try to access the /proc/sys/fs/binfmt_misc directory... systemd tries to mount it and it fails. I then run the audit_rules_privileged_commands rule and see the error. In most cases the error appears only once and then the /proc/sys/fs/binfmt_misc gets mounted and the error no longer appears. This for some reason did not happen in this session. This is shown in the script_1 file.
The script_2 file shows how I run the scan for the whole anssi_bp28_high profile. The directory gets mounted it seems, at least it can be accessed from shell...
Note that this session is recorded on RHEL8
script1.log
script2.log
.

[jan-cerny]

Does that mean that oscap has problems with accessing automounts?

[jan-cerny]

We had multiple issues in past with auto mounts: OpenSCAP/openscap#1329 OpenSCAP/openscap#1342

[vojtapolasek]

Just a note: This happens on a RHEL8 system after I did

oscap xccdf eval --remediate --profile anssi_bp28_high ssg-rhel8-ds.xml
shutdown -r now
~~~

[mildas]

Should we skip audit_rules_privileged_command evaluation in test until it gets fixed in scanner?

[marcusburghardt]

Should we skip audit_rules_privileged_command evaluation in test until it gets fixed in scanner?

I believe the suggestion from @jan-cerny , to remove the /proc by updating the regex would be a safer temporary workaround. Once the CI is stable again we can investigate the behavior of this mount point controlled by systemd. @jan-cerny , do you plan to propose a PR for this workaround? I would also be happy to review/test it.

[vojtapolasek]

Another note - if you perform evaluation with the full profile (not using ds_unselect_rules script), you will encounter similar error message at multiple rules but it is about polyinstantiated temporary directories in /tmp/tmp-inst.
I plan to investigate which rule causes this behavior. I suspect it is some new rule which is used only in ANSSI2.0 and not in another profile because I don't remember seeing this error before.

[jan-cerny]

I was able to reproduce it locally on a RHEL 8.8 Virtual machine. I performed scan, remediation, reboot and another scan. The error message appeared during the scan after the reboot. The error message appeared twice, first during evaluation of rule xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands and the second during evaluation of rule xccdf_org.ssgproject.content_rule_mount_option_nodev_nonroot_local_partitions.

Then, I checked the status of the systemd automount and mount units that are responsible for the pseudo file system mounted at our problematic path. I find the output interesting, because both systemd units are in a failed state:

[user@localhost ~]$ sudo systemctl status proc-sys-fs-binfmt_misc.automount
[sudo] password for user: 
● proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point
   Loaded: loaded (/usr/lib/systemd/system/proc-sys-fs-binfmt_misc.automount; static; vendor preset: disabled)
   Active: failed (Result: mount-start-limit-hit) since Tue 2023-04-18 09:38:44 CEST; 5min ago
    Where: /proc/sys/fs/binfmt_misc
     Docs: https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html
           https://www.freedesktop.org/wiki/Software/systemd/APIFileSystems

Apr 18 09:38:41 localhost.localdomain systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 6088 (probe_worker)
Apr 18 09:38:43 localhost.localdomain systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 6386 (probe_worker)
Apr 18 09:38:43 localhost.localdomain systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 6398 (probe_worker)
Apr 18 09:38:43 localhost.localdomain systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 6410 (probe_worker)
Apr 18 09:38:44 localhost.localdomain systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 6422 (probe_worker)
Apr 18 09:38:44 localhost.localdomain systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 6434 (probe_worker)
Apr 18 09:38:44 localhost.localdomain systemd[1]: proc-sys-fs-binfmt_misc.automount: Failed with result 'mount-start-limit-hit'.
[user@localhost ~]$ sudo systemctl status proc-sys-fs-binfmt_misc.mount
● proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System
   Loaded: loaded (/usr/lib/systemd/system/proc-sys-fs-binfmt_misc.mount; static; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2023-04-18 09:38:44 CEST; 6min ago
    Where: /proc/sys/fs/binfmt_misc
     What: binfmt_misc
     Docs: https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html
           https://www.freedesktop.org/wiki/Software/systemd/APIFileSystems

Apr 18 09:38:44 localhost.localdomain systemd[1]: Mounting Arbitrary Executable File Formats File System...
Apr 18 09:38:44 localhost.localdomain mount[6423]: mount: /proc/sys/fs/binfmt_misc: unknown filesystem type 'binfmt_misc'.
Apr 18 09:38:44 localhost.localdomain systemd[1]: proc-sys-fs-binfmt_misc.mount: Mount process exited, code=exited status=32
Apr 18 09:38:44 localhost.localdomain systemd[1]: proc-sys-fs-binfmt_misc.mount: Failed with result 'exit-code'.
Apr 18 09:38:44 localhost.localdomain systemd[1]: Failed to mount Arbitrary Executable File Formats File System.
Apr 18 09:38:44 localhost.localdomain systemd[1]: proc-sys-fs-binfmt_misc.mount: Start request repeated too quickly.
Apr 18 09:38:44 localhost.localdomain systemd[1]: proc-sys-fs-binfmt_misc.mount: Failed with result 'exit-code'.
Apr 18 09:38:44 localhost.localdomain systemd[1]: Failed to mount Arbitrary Executable File Formats File System.

[vojtapolasek]

With help of @jan-cerny's findings, I managed to found the cause of the problem. And the winner is... the rule sebool_secure_mode_insmod. It effectively blocks loading of modules by processes. I suggest we do the same as with the rule sysctl_kernel_modules_disabled - disable the Bash remediation because it will cause the some trouble. Opinions@ @yuumasato

[vojtapolasek]

After further discussions, I decided to suggest a PR which skips the /proc directory for audit_rules_privileged_commands rule. Removing remediation of the sebool_secure_mode_insmod would solve the problem, but the rule is just doing its job and it is doing it apparently well.