If we use a .sh or .py(-rwxr-xr--.) has defined #!/bin/sh or #!/usr/bin/python3 as the pid=1 of the container, starting the container with a normal user without permissions will report an error "permission denied"

[zzzzzzzzzy9]

Description

While golang >= 1.20, If we use .sh and .py as the pid=1 of the container, starting the container with a normal user without permissions will report an error "permission denied".

But If the golang <1.20, If we use .sh and .py as the pid=1 of the container, starting the container with a normal user will start the .py or .sh success.

        // Check for the arg before waiting to make sure it exists and it is
	// returned as a create time error.
	name, err := exec.LookPath(l.config.Args[0])
	if err != nil {
		return err
	}

https://github.com/opencontainers/runc/blob/main/libcontainer/standard_init_linux.go#L202

This error return by exec.LookPath. In golang 1.20, golang/go@2b8f214
I can understand that this commit introduces Eaccess.

But if we comment out this line, runc can start a .sh or .py script without permissions(we should define #!/bin/sh or #!/bin/python3 in script).
Actually in scripts, even if the file's mode is 754(-rwxr-xr--.), we can also call /bin/bash .sh or /usr/bin/python3 .py to run this script successfully without permision.

I think this is because syscall.Exec will call /bin/sh or /usr/bin/python3 as pid=1, and then, call /bin/sh .sh or /usr/bin/python3 .py, this is permitted.(In fact, I used the ps command to check process relationships, and indeed started /usr/bin/bash before starting the script.) But I cannot do this outside runc, I can only do this by commenting out the lookpath line and the Eaccess line in runc, is there anyone can explain this?

But now, After runc is compiled by golang>=1.20, this will not succeed anymore.

Steps to reproduce the issue

1. dockerfile just like this:

FROM centos
COPY sleep.sh /
COPY sleep.py /
RUN useradd -u 1000 test

sleep.sh and sleep.py mode is 754, just means -rwxr-xr--. sleep.sh and sleep.py like this:

#!/bin/sh
echo "sleep"
sleep 1000

#!/usr/bin/python3
import time
time.sleep(1000)
print("hello world")

2. docker build -t centos:sleep .
3. use the older version runc compiled by golang<1.20
4. docker run -id -u 1000 centos:sleep ./sleep.sh
5. use the up-to-date runc compiled by golang>=1.20
6. docker run -id -u 1000 centos:sleep ./sleep.sh

Describe the results you received and expected

If runc is compiled by golang>=1.20, it will return "permission denied".
If runc is compiled by golang<1.20, it will success.

What version of runc are you using?

1. up-to-date runc
2. runc compiled by golang<1.20 and delete

        // exec.LookPath in Go < 1.20 might return no error for an executable
	// residing on a file system mounted with noexec flag, so perform this
	// extra check now while we can still return a proper error.
	// TODO: remove this once go < 1.20 is not supported.
	if err := eaccess(name); err != nil {
		return &os.PathError{Op: "eaccess", Path: name, Err: err}
	}

Host OS information

all centos

Host kernel information

4.18

[cyphar]

I've tried to reproduce the issue you described (removing the eaccess check and then building runc with Go 1.19), and you still get permission denied when trying to execute the script. Given how execve works (the kernel handles everything for us), I would be very surprised if a change in the Go stdlib version would cause this issue.

The way permissions work with #! is that the kernel tries to execute the file (triggering a MAY_EXEC check on the original file) and that then leads to the binfmt_script code reading the interpreter line and then executing the interpreter (which triggers a MAY_EXEC check on the interpreter). You can easily verify this behaviour with:

% echo '#!/bin/echo' > script
% ./script
zsh: permission denied: ./script
% chmod +x ./script
% ./script
./script

And you can verify that the error is really coming directly from exec, meaning the kernel is refusing the access:

% strace -e execve ./script
execve("./script", ["./script"], 0x7ffdd15d1b90 /* 104 vars */) = -1 EACCES (Permission denied)
strace: exec: Permission denied
+++ exited with 1 +++

All of this means that syscall.Exec doesn't (and shouldn't!) parse the interpreter line of the script. You can verify the same thing happens in runc:

% strace -f -e execve ./runc run -b bundle ctr
execve("./runc", ["./runc", "run", "-b", "bundle", "ctr"], 0x7ffed7c829b8 /* 13 vars */) = 0
strace: Process 18784 attached
strace: Process 18785 attached
strace: Process 18786 attached
strace: Process 18787 attached
strace: Process 18788 attached
strace: Process 18789 attached
strace: Process 18790 attached
strace: Process 18791 attached
[pid 18791] execve("/proc/self/exe", ["./runc", "init"], 0xc000226000 /* 8 vars */) = 0
strace: Process 18792 attached
[pid 18789] --- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=18783, si_uid=0} ---
[pid 18789] --- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=18783, si_uid=0} ---
strace: Process 18793 attached
strace: Process 18794 attached
[pid 18793] +++ exited with 0 +++
[pid 18789] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=18793, si_uid=0, si_status=0, si_utime=0, si_stime=0} ---
[pid 18791] +++ exited with 0 +++
[pid 18789] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=18791, si_uid=0, si_status=0, si_utime=0, si_stime=0} ---
strace: Process 18795 attached
strace: Process 18796 attached
strace: Process 18797 attached
[pid 18794] --- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=1, si_uid=0} ---
strace: Process 18798 attached
strace: Process 18799 attached
strace: Process 18800 attached
[pid 18789] --- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=18783, si_uid=0} ---
[pid 18794] execve("/script", ["/script"], 0xc000024660 /* 3 vars */) = -1 EACCES (Permission denied)
[pid 18799] +++ exited with 1 +++
[pid 18798] +++ exited with 1 +++
[pid 18797] +++ exited with 1 +++
[pid 18796] +++ exited with 1 +++
[pid 18795] +++ exited with 1 +++
exec /script: permission denied
[pid 18794] +++ exited with 1 +++
[pid 18786] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=18794, si_uid=1000, si_status=1, si_utime=0, si_stime=1 /* 0.01 s */} ---
[pid 18800] +++ exited with 1 +++
[pid 18792] +++ exited with 1 +++
[pid 18790] +++ exited with 1 +++
[pid 18789] +++ exited with 1 +++
[pid 18788] +++ exited with 1 +++
[pid 18787] +++ exited with 1 +++
[pid 18786] +++ exited with 1 +++
[pid 18785] +++ exited with 1 +++
[pid 18784] +++ exited with 1 +++
+++ exited with 1 +++

The key line being:

[pid 18794] execve("/script", ["/script"], 0xc000024660 /* 3 vars */) = -1 EACCES (Permission denied)

We are just passing the script path to the kernel, and it handles everything else.

[zzzzzzzzzy9]

Actually I started the container through docker.
If I use runc spec to generate a config.json file, and then start the container will report the error "permission denied".
I then looked at the difference between docker and runc. I found that there is a parameter in capability CAP_DAC_ OVERRIDE affects this phenomenon. If we use runc, adding CAP_DAC_ OVERRIDE to the config.json file can successfully pull up permissionless .sh, which may be the cause of this problem.
@cyphar

[cyphar]

Ah yeah, CAP_DAC_OVERRIDE would bypass the checks, however the faccessat2(AT_EACCESS) check also obeys CAP_DAC_OVERRIDE (EDIT: I just noticed your kernel is quite old and so probably doesn't have faccessat2 support unless Red Hat backported it).

There was a bug with how this check was handled by the standard library when CAP_DAC_OVERRIDE was set, but this was fixed in Go 1.20.2 -- this was an issue on older kernels that don't have faccessat2 support. What version of Go are you using?

We fixed this issue and added tests for it in #3753 (which was released in runc 1.1.6) and this bug seems like a duplicate of #3715 (which we've long-since fixed AFAIK).

[zzzzzzzzzy9]

I'm using golang1.20.5.

exec.LookPath will check the capabilities, it works fine. I have seen if err := system.Eaccess(name); err != nil { in runc will check the CAP_DAC_OVERRIDE, this work fine too.
But, syscall.Exec doesn't seems to check the capabilities. I have tried in kernel-5.10.134, syscall.Exec won't return permission denied while using CAP_DAC_OVERRIDE. Is this a problem of golang syscall.Exec? If not, if err := system. Eaccess(name); err != nil { in runc is it redundant?
Whether permission-check should be given to syscall. Exec to execute, instead of reporting an error in the exec.LookPath? (after all, CAP_DAC_OVERRIDE is reasonable for syscall.Exec)

@cyphar

[kolyshkin]

@zzyyzte so, do you still see this bug? We believe it was fixed in runc 1.1.6 (unless it is compiled by go > 1.20.2).

if err := system. Eaccess(name); err != nil { in runc is it redundant?

Now it is (and it was dropped in 8491d33 / 840b953).

[kolyshkin]

Whether permission-check should be given to syscall. Exec to execute, instead of reporting an error in the exec.LookPath?

This was done to fix #3520. Problem is, when we call syscall.Exec, it is too late to report any errors in a standard way, since we have closed all means of reporting.

This is why we do this check early -- to be able to report an error.

[kolyshkin]

Closing as fixed; @zzyyzte feel free to let us know if the issue is still not fixed