Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid 32-bit libraries when IA32_EMULATION is off #2190

Closed
LaszloGombos opened this issue Feb 8, 2023 · 5 comments · Fixed by #2193
Closed

Avoid 32-bit libraries when IA32_EMULATION is off #2190

LaszloGombos opened this issue Feb 8, 2023 · 5 comments · Fixed by #2193
Labels
bug Our bugs

Comments

@LaszloGombos
Copy link
Collaborator

LaszloGombos commented Feb 8, 2023

When IA32_EMULATION is off on a 64-bit only x86-64 configuration, 32-bit binaries would likely break the boot process.

The exact configuration for this error is not yet confirmed, but "32-bit libraries that don't work end up also hiding 64-bit ones that do work" when binaries from the host are copied to the initramfs. Confirmed on F37, but distribution is likely irrelevant.

@LaszloGombos LaszloGombos added the bug Our bugs label Feb 8, 2023
@LaszloGombos
Copy link
Collaborator Author

From @aafeijoo-suse

Debugging the ldd script, the problem is in this line:

RTLDLIST="/lib/ld-linux.so.2 /lib64/ld-linux-x86-64.so.2 /libx32/ld-linux-x32.so.2"

It's picking the 32-bit library just because it exists on the system,
then it fails to execute.

After removing /lib/ld-linux.so.2 from the RTLDLIST variable, the initrd
should contain all the necessary libraries.

@torvalds
Copy link

torvalds commented Feb 8, 2023

So since this actually impacts even installing a distro kernel, and I had an example of both a good and a bad case in the form of a old initramfs images that had been installed before and while running a 64-bit only kernel, I did this:

  zcat < initramfs-6.1.6-100.fc36.x86_64.img | cpio -t | sed 's/6.1.6-100.fc36.x86_64/KERNELVERSION/' > old
  zcat < initramfs-6.1.9-100.fc36.x86_64.img | cpio -t | sed 's/6.1.9-100.fc36.x86_64/KERNELVERSION/' > new
  diff -u old new | less -S

to just extract a file lists and compare them (that 'sed' part is to just make the different module paths compare the same, of course).

In the above, the "old" is a good install that was done before I had turned off support for 32-bit compat mode, and "new" is the file list of a kernel install that had been done on the exact same system, just with a kernel that no lonegr ran 32-bit binaries.

Interestingly, the difference is almost entirely that "new" is missing a lot of 64-bit libraries in /usr/lib64, so the diff starts out like this:

--- old 2023-02-08 08:02:59.645586945 -0800
+++ new 2023-02-08 08:03:17.541428445 -0800
@@ -1924,7 +1924,6 @@
 usr/lib/systemd/catalog
 usr/lib/systemd/catalog/dbus-broker-launch.catalog
 usr/lib/systemd/catalog/dbus-broker.catalog
-usr/lib/systemd/libsystemd-shared-250.so
 usr/lib/systemd/network
 usr/lib/systemd/network/99-default.link
 usr/lib/systemd/system
@@ -2130,175 +2129,21 @@
 usr/lib64/device-mapper/libdevmapper-event-lvm2thin.so
 usr/lib64/device-mapper/libdevmapper-event-lvm2vdo.so
 usr/lib64/ld-linux-x86-64.so.2
-usr/lib64/libacl.so.1
-usr/lib64/libacl.so.1.1.2301
-usr/lib64/libaio.so.1
-usr/lib64/libaio.so.1.0.1
-usr/lib64/libargon2.so.1
-usr/lib64/libattr.so.1
-usr/lib64/libattr.so.1.1.2501
-usr/lib64/libaudit.so.1
... snip snip ...
 usr/lib64/libz.so
 usr/lib64/libz.so.1
 usr/lib64/libz.so.1.2.11
-usr/lib64/libzstd.so
-usr/lib64/libzstd.so.1
-usr/lib64/libzstd.so.1.5.2
 usr/lib64/opensc-pkcs11.so
 usr/lib64/plymouth
 usr/lib64/plymouth/details.so

and the snipped out part doesn't really get any more interesting.

There are no actual 32-bit binary files in the good case, with the following returning an empty set of files:

   mkdir good-filesystem
   cd good-filesystem/
   zcat < ../initramfs-6.1.6-100.fc36.x86_64.img | cpio -i
   find . -type f | xargs file | grep 32-bit

so the failure is not that some 32-bit binary was used, the failure really seems to be that just the existence of 32-bit binaries that were not used then ended up confusing some logic enough to not include various 64-bit libraries.

Some fallout from the ldd failure does seem very likely.

@torvalds
Copy link

torvalds commented Feb 8, 2023

Just to clarify: everything works for me now, because I blew away all my old 32-bit binaries on this machine, so now when I run dracut on a kernel that doesn't have support for 32-bit compat more, it all works correctly.

So this is not a high priority, but it was very confusing, and it might bite somebody else at some future point where people start deciding more widely that 32-bit compat support is no longer interesting. Having left-over 32-bit binaries that aren't actually used any more then causes this odd failure on installing a new kernel, when everything else seems to work just fine.

@aafeijoo-suse
Copy link
Member

so the failure is not that some 32-bit binary was used, the failure really seems to be that just the existence of 32-bit binaries that were not used then ended up confusing some logic enough to not include various 64-bit libraries.

Some fallout from the ldd failure does seem very likely.

Yes, I came to the same conclusion yesterday:

# zgrep CONFIG_IA32_EMULATION /proc/config.gz 
# CONFIG_IA32_EMULATION is not set
# dracut -f --stdlog 3
# ls -la /boot/initrd-$(uname -r)*
-rw------- 1 root root 77114628 Feb  8 10:29 /boot/initrd-6.2.0-rc4-1-default
-rw------- 1 root root 77314190 Feb  8 10:30 /boot/initrd-6.2.0-rc4-1-default.bak
# zypper in glibc-32bit
...
# dracut -f --stdlog 3
/usr/bin/ldd: line 160: /lib/ld-linux.so.2: cannot execute binary file: Exec format error
# ls -la /boot/initrd-$(uname -r)*
-rw------- 1 root root 69412756 Feb  8 10:32 /boot/initrd-6.2.0-rc4-1-default
-rw------- 1 root root 77314190 Feb  8 10:30 /boot/initrd-6.2.0-rc4-1-default.bak

This new initrd fails to boot the system. After manually removing /lib/ld-linux.so.2 from the RTLDLIST variable in /usr/bin/ldd, the error goes away and there are no missing libraries in the newly generated initrds.

@aafeijoo-suse
Copy link
Member

Although the ldd failure is not critical, because after failing with /lib/ld-linux.so.2, it attempts /lib64/ld-linux-x86-64.so.2 and produces the expected output:

# ldd /lib64/libfido2.so.1.12.0
/bin/ldd: line 162: /lib/ld-linux.so.2: cannot execute binary file: Exec format error
	linux-vdso.so.1 (0x00007ffcb8bd9000)
	libcbor.so.0.9 => /lib64/libcbor.so.0.9 (0x00007f7fe8a51000)
	libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007f7fe8600000)
	libudev.so.1 => /lib64/libudev.so.1 (0x00007f7fe85d2000)
	libhidapi-hidraw.so.0 => /lib64/libhidapi-hidraw.so.0 (0x00007f7fe8a48000)
	libz.so.1 => /lib64/libz.so.1 (0x00007f7fe85b8000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f7fe83bd000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f7fe8aab000)

But dracut-install is breaking here:

if (strstr(buf, "cannot execute binary file"))
break;

So, is our bug.

aafeijoo-suse added a commit to aafeijoo-suse/dracut that referenced this issue Feb 9, 2023
…inary file"

When the kernel is compiled without IA32_EMULATION and the glibc 32-bit library
is installed on the system, `ldd` prints the following output:

```
> ldd /usr/lib64/libfido2.so.1.12.0
/bin/ldd: line 162: /lib/ld-linux.so.2: cannot execute binary file: Exec format error
	linux-vdso.so.1 (0x00007ffd627fa000)
	libcbor.so.0.9 => /lib64/libcbor.so.0.9 (0x00007f18d799f000)
	libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007f18d7400000)
	libudev.so.1 => /lib64/libudev.so.1 (0x00007f18d7971000)
	libhidapi-hidraw.so.0 => /lib64/libhidapi-hidraw.so.0 (0x00007f18d7968000)
	libz.so.1 => /lib64/libz.so.1 (0x00007f18d794e000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f18d7205000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f18d79f9000)
> echo $?
0
```

The `ldd` script uses the following code to resolve dependencies:

```
RTLDLIST="/lib/ld-linux.so.2 /lib64/ld-linux-x86-64.so.2 /libx32/ld-linux-x32.so.2"
...
RTLD=
ret=1
for rtld in ${RTLDLIST}; do
  if test -x $rtld; then
    verify_out=`${rtld} --verify "$file"`
    ret=$?
    case $ret in
      [02]) RTLD=${rtld}; break;;
    esac
  fi
done
```

So, if the 32-bit library fails, the 64-bit library may work, so don't stop
parsing the `ldd` output unconditionally when the message "cannot execute binary
file" is printed.

Fixes issue dracutdevs#2190
aafeijoo-suse added a commit to aafeijoo-suse/dracut that referenced this issue Feb 9, 2023
…inary file"

When the kernel is compiled without IA32_EMULATION and the glibc 32-bit library
is installed on the system, `ldd` prints the following output:

```
> ldd /usr/lib64/libfido2.so.1.12.0
/bin/ldd: line 162: /lib/ld-linux.so.2: cannot execute binary file: Exec format error
	linux-vdso.so.1 (0x00007ffd627fa000)
	libcbor.so.0.9 => /lib64/libcbor.so.0.9 (0x00007f18d799f000)
	libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007f18d7400000)
	libudev.so.1 => /lib64/libudev.so.1 (0x00007f18d7971000)
	libhidapi-hidraw.so.0 => /lib64/libhidapi-hidraw.so.0 (0x00007f18d7968000)
	libz.so.1 => /lib64/libz.so.1 (0x00007f18d794e000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f18d7205000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f18d79f9000)
> echo $?
0
```

The `ldd` script uses the following code to resolve dependencies:

```
RTLDLIST="/lib/ld-linux.so.2 /lib64/ld-linux-x86-64.so.2 /libx32/ld-linux-x32.so.2"
...
RTLD=
ret=1
for rtld in ${RTLDLIST}; do
  if test -x $rtld; then
    verify_out=`${rtld} --verify "$file"`
    ret=$?
    case $ret in
      [02]) RTLD=${rtld}; break;;
    esac
  fi
done
```

So, if the 32-bit library fails, the 64-bit library may work, so don't stop
parsing the `ldd` output unconditionally when the message "cannot execute binary
file" is printed.

Fixes issue dracutdevs#2190
johannbg pushed a commit that referenced this issue Feb 13, 2023
…inary file"

When the kernel is compiled without IA32_EMULATION and the glibc 32-bit library
is installed on the system, `ldd` prints the following output:

```
> ldd /usr/lib64/libfido2.so.1.12.0
/bin/ldd: line 162: /lib/ld-linux.so.2: cannot execute binary file: Exec format error
	linux-vdso.so.1 (0x00007ffd627fa000)
	libcbor.so.0.9 => /lib64/libcbor.so.0.9 (0x00007f18d799f000)
	libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007f18d7400000)
	libudev.so.1 => /lib64/libudev.so.1 (0x00007f18d7971000)
	libhidapi-hidraw.so.0 => /lib64/libhidapi-hidraw.so.0 (0x00007f18d7968000)
	libz.so.1 => /lib64/libz.so.1 (0x00007f18d794e000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f18d7205000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f18d79f9000)
> echo $?
0
```

The `ldd` script uses the following code to resolve dependencies:

```
RTLDLIST="/lib/ld-linux.so.2 /lib64/ld-linux-x86-64.so.2 /libx32/ld-linux-x32.so.2"
...
RTLD=
ret=1
for rtld in ${RTLDLIST}; do
  if test -x $rtld; then
    verify_out=`${rtld} --verify "$file"`
    ret=$?
    case $ret in
      [02]) RTLD=${rtld}; break;;
    esac
  fi
done
```

So, if the 32-bit library fails, the 64-bit library may work, so don't stop
parsing the `ldd` output unconditionally when the message "cannot execute binary
file" is printed.

Fixes issue #2190
aafeijoo-suse added a commit to aafeijoo-suse/dracut that referenced this issue Jun 15, 2023
…inary file"

When the kernel is compiled without IA32_EMULATION and the glibc 32-bit library
is installed on the system, `ldd` prints the following output:

```
> ldd /usr/lib64/libfido2.so.1.12.0
/bin/ldd: line 162: /lib/ld-linux.so.2: cannot execute binary file: Exec format error
	linux-vdso.so.1 (0x00007ffd627fa000)
	libcbor.so.0.9 => /lib64/libcbor.so.0.9 (0x00007f18d799f000)
	libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007f18d7400000)
	libudev.so.1 => /lib64/libudev.so.1 (0x00007f18d7971000)
	libhidapi-hidraw.so.0 => /lib64/libhidapi-hidraw.so.0 (0x00007f18d7968000)
	libz.so.1 => /lib64/libz.so.1 (0x00007f18d794e000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f18d7205000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f18d79f9000)
> echo $?
0
```

The `ldd` script uses the following code to resolve dependencies:

```
RTLDLIST="/lib/ld-linux.so.2 /lib64/ld-linux-x86-64.so.2 /libx32/ld-linux-x32.so.2"
...
RTLD=
ret=1
for rtld in ${RTLDLIST}; do
  if test -x $rtld; then
    verify_out=`${rtld} --verify "$file"`
    ret=$?
    case $ret in
      [02]) RTLD=${rtld}; break;;
    esac
  fi
done
```

So, if the 32-bit library fails, the 64-bit library may work, so don't stop
parsing the `ldd` output unconditionally when the message "cannot execute binary
file" is printed.

Fixes issue dracutdevs#2190

(cherry picked from commit 9a531ca)
aafeijoo-suse added a commit to aafeijoo-suse/dracut that referenced this issue Jun 26, 2023
…inary file"

When the kernel is compiled without IA32_EMULATION and the glibc 32-bit library
is installed on the system, `ldd` prints the following output:

```
> ldd /usr/lib64/libfido2.so.1.12.0
/bin/ldd: line 162: /lib/ld-linux.so.2: cannot execute binary file: Exec format error
	linux-vdso.so.1 (0x00007ffd627fa000)
	libcbor.so.0.9 => /lib64/libcbor.so.0.9 (0x00007f18d799f000)
	libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007f18d7400000)
	libudev.so.1 => /lib64/libudev.so.1 (0x00007f18d7971000)
	libhidapi-hidraw.so.0 => /lib64/libhidapi-hidraw.so.0 (0x00007f18d7968000)
	libz.so.1 => /lib64/libz.so.1 (0x00007f18d794e000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f18d7205000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f18d79f9000)
> echo $?
0
```

The `ldd` script uses the following code to resolve dependencies:

```
RTLDLIST="/lib/ld-linux.so.2 /lib64/ld-linux-x86-64.so.2 /libx32/ld-linux-x32.so.2"
...
RTLD=
ret=1
for rtld in ${RTLDLIST}; do
  if test -x $rtld; then
    verify_out=`${rtld} --verify "$file"`
    ret=$?
    case $ret in
      [02]) RTLD=${rtld}; break;;
    esac
  fi
done
```

So, if the 32-bit library fails, the 64-bit library may work, so don't stop
parsing the `ldd` output unconditionally when the message "cannot execute binary
file" is printed.

Fixes issue dracutdevs#2190

(cherry picked from commit 9a531ca)

bsc#1212662
aafeijoo-suse added a commit to aafeijoo-suse/dracut that referenced this issue Jun 26, 2023
…inary file"

When the kernel is compiled without IA32_EMULATION and the glibc 32-bit library
is installed on the system, `ldd` prints the following output:

```
> ldd /usr/lib64/libfido2.so.1.12.0
/bin/ldd: line 162: /lib/ld-linux.so.2: cannot execute binary file: Exec format error
	linux-vdso.so.1 (0x00007ffd627fa000)
	libcbor.so.0.9 => /lib64/libcbor.so.0.9 (0x00007f18d799f000)
	libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007f18d7400000)
	libudev.so.1 => /lib64/libudev.so.1 (0x00007f18d7971000)
	libhidapi-hidraw.so.0 => /lib64/libhidapi-hidraw.so.0 (0x00007f18d7968000)
	libz.so.1 => /lib64/libz.so.1 (0x00007f18d794e000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f18d7205000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f18d79f9000)
> echo $?
0
```

The `ldd` script uses the following code to resolve dependencies:

```
RTLDLIST="/lib/ld-linux.so.2 /lib64/ld-linux-x86-64.so.2 /libx32/ld-linux-x32.so.2"
...
RTLD=
ret=1
for rtld in ${RTLDLIST}; do
  if test -x $rtld; then
    verify_out=`${rtld} --verify "$file"`
    ret=$?
    case $ret in
      [02]) RTLD=${rtld}; break;;
    esac
  fi
done
```

So, if the 32-bit library fails, the 64-bit library may work, so don't stop
parsing the `ldd` output unconditionally when the message "cannot execute binary
file" is printed.

Fixes issue dracutdevs#2190

(cherry picked from commit 9a531ca)

bsc#1212662
pvalena pushed a commit to pvalena/dracut that referenced this issue Jul 23, 2023
…inary file"

When the kernel is compiled without IA32_EMULATION and the glibc 32-bit library
is installed on the system, `ldd` prints the following output:

```
> ldd /usr/lib64/libfido2.so.1.12.0
/bin/ldd: line 162: /lib/ld-linux.so.2: cannot execute binary file: Exec format error
	linux-vdso.so.1 (0x00007ffd627fa000)
	libcbor.so.0.9 => /lib64/libcbor.so.0.9 (0x00007f18d799f000)
	libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007f18d7400000)
	libudev.so.1 => /lib64/libudev.so.1 (0x00007f18d7971000)
	libhidapi-hidraw.so.0 => /lib64/libhidapi-hidraw.so.0 (0x00007f18d7968000)
	libz.so.1 => /lib64/libz.so.1 (0x00007f18d794e000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f18d7205000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f18d79f9000)
> echo $?
0
```

The `ldd` script uses the following code to resolve dependencies:

```
RTLDLIST="/lib/ld-linux.so.2 /lib64/ld-linux-x86-64.so.2 /libx32/ld-linux-x32.so.2"
...
RTLD=
ret=1
for rtld in ${RTLDLIST}; do
  if test -x $rtld; then
    verify_out=`${rtld} --verify "$file"`
    ret=$?
    case $ret in
      [02]) RTLD=${rtld}; break;;
    esac
  fi
done
```

So, if the 32-bit library fails, the 64-bit library may work, so don't stop
parsing the `ldd` output unconditionally when the message "cannot execute binary
file" is printed.

Fixes issue dracutdevs#2190
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Our bugs
Projects
None yet
3 participants