ONBUILD duplicate executions for nested multi-stage builds

[jfine]

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

Please let me know if this issue should be created elsewhere!

From <= 27.3.1 when utilizing nested multi-stage builds the ONBUILD command would only be executed once, similar to what happens when building off of nested images in which the grandparent image has ONBUILD commands. The documentation (https://docs.docker.com/reference/dockerfile/#onbuild) also states...

Triggers are cleared from the final image after being executed. In other words they aren't inherited by "grand-children" builds.

...however >= 27.4.0 multi-stage builds appear to be executing the ONBUILD command for every nested stage and are no longer cleared after being executed.

Expected behaviour

When utilizing multi-stage builds the ONBUILD triggers should be cleared and not inherited by grand-children builds, similar to when separate images are used (instead of stages)...

[+] Building 1.2s (6/6) FINISHED                                                                   docker:default
 => [internal] load build definition from Dockerfile.multistep                                               0.0s
 => => transferring dockerfile: 97B                                                                          0.0s
 => [internal] load metadata for docker.io/library/onbuild-base:latest                                       0.0s
 => [internal] load .dockerignore                                                                            0.0s
 => => transferring context: 2B                                                                              0.0s
 => CACHED [step-1 1/1] FROM docker.io/library/onbuild-base:latest                                           0.0s
 => [step-1 2/1] RUN groupadd testgroup                                                                      0.8s
 => exporting to image                                                                                       0.2s
 => => exporting layers                                                                                      0.2s
 => => writing image sha256:56980b1281e123e8b7c985530a49f6b9b1bd90bc5ef3a5d2d9f2889f102f1308                 0.0s
 => => naming to docker.io/library/multistep                                                                 0.0s

Actual behaviour

[+] Building 1.3s (6/6) FINISHED                                                                   docker:default
 => [internal] load build definition from Dockerfile.multistep                                               0.0s
 => => transferring dockerfile: 97B                                                                          0.0s
 => [internal] load metadata for docker.io/library/onbuild-base:latest                                       0.0s
 => [internal] load .dockerignore                                                                            0.0s
 => => transferring context: 2B                                                                              0.0s
 => CACHED [step-1 1/1] FROM docker.io/library/onbuild-base:latest                                           0.0s
 => [step-1 2/2] ONBUILD RUN groupadd testgroup                                                              0.7s
 => ERROR [step-2 1/1] ONBUILD RUN groupadd testgroup                                                        0.3s
------                                                                                                            
 > [step-2 1/1] ONBUILD RUN groupadd testgroup:
0.290 groupadd: group 'testgroup' already exists
------
Dockerfile.multistep:2
--------------------
   1 |     FROM onbuild-base AS step-1
   2 | >>> FROM step-1 AS step-2
   3 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c groupadd testgroup" did not complete successfully: exit code: 9

Buildx version

github.com/docker/buildx v0.19.3 48d6a39

Docker info

Client: Docker Engine - Community
 Version:    27.4.0
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.19.3
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.32.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 27.4.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: false
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 88bf19b2105c8b17560993bee28a01ddc2f97182
 runc version: v1.2.2-0-g7cb3632
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-1018-azure
 Operating System: Ubuntu 24.04.1 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 7.703GiB
 Name: docker-test
 ID: e82a175f-2360-4d15-8131-ce72172bdcac
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

Builders list

NAME/NODE     DRIVER/ENDPOINT   STATUS    BUILDKIT   PLATFORMS
default*      docker                                 
 \_ default    \_ default       running   v0.17.3    linux/amd64 (+4), linux/386

Configuration

Dockerfile.onbuild-base:

FROM ubuntu
ONBUILD RUN groupadd testgroup

Dockerfile.multistep:

FROM onbuild-base AS step-1
FROM step-1 AS step-2

Dockerfiles.zip

$ docker build --tag onbuild-base --file Dockerfile.onbuild-base .
$ docker build --tag multistep --file Dockerfile.multistep .

Build logs

$ docker build --tag onbuild-base --file Dockerfile.onbuild-base .
[+] Building 2.6s (5/5) FINISHED                                                                   docker:default
 => [internal] load build definition from Dockerfile.onbuild-base                                            0.1s
 => => transferring dockerfile: 93B                                                                          0.0s
 => [internal] load metadata for docker.io/library/ubuntu:latest                                             0.3s
 => [internal] load .dockerignore                                                                            0.1s
 => => transferring context: 2B                                                                              0.0s
 => [1/1] FROM docker.io/library/ubuntu:latest@sha256:80dd3c3b9c6cecb9f1667e9290b3bc61b78c2678c02cbdae5f0fe  1.9s
 => => resolve docker.io/library/ubuntu:latest@sha256:80dd3c3b9c6cecb9f1667e9290b3bc61b78c2678c02cbdae5f0fe  0.0s
 => => sha256:80dd3c3b9c6cecb9f1667e9290b3bc61b78c2678c02cbdae5f0fea92cc6734ab 6.69kB / 6.69kB               0.0s
 => => sha256:6e75a10070b0fcb0bead763c5118a369bc7cc30dfc1b0749c491bbb21f15c3c7 424B / 424B                   0.0s
 => => sha256:b1d9df8ab81559494794e522b380878cf9ba82d4c1fb67293bcf931c3aa69ae4 2.30kB / 2.30kB               0.0s
 => => sha256:de44b265507ae44b212defcb50694d666f136b35c1090d9709068bc861bb2d64 29.75MB / 29.75MB             0.4s
 => => extracting sha256:de44b265507ae44b212defcb50694d666f136b35c1090d9709068bc861bb2d64                    1.3s
 => exporting to image                                                                                       0.0s
 => => exporting layers                                                                                      0.0s
 => => writing image sha256:ae666a0d2fbe91d4e74256560e9373b77e2f2142a25fa0a48d666435380c3c2a                 0.0s
 => => naming to docker.io/library/onbuild-base                                                              0.0s
$ docker build --tag multistep --file Dockerfile.multistep .
[+] Building 1.3s (6/6) FINISHED                                                                   docker:default
 => [internal] load build definition from Dockerfile.multistep                                               0.0s
 => => transferring dockerfile: 97B                                                                          0.0s
 => [internal] load metadata for docker.io/library/onbuild-base:latest                                       0.0s
 => [internal] load .dockerignore                                                                            0.0s
 => => transferring context: 2B                                                                              0.0s
 => CACHED [step-1 1/1] FROM docker.io/library/onbuild-base:latest                                           0.0s
 => [step-1 2/2] ONBUILD RUN groupadd testgroup                                                              0.7s
 => ERROR [step-2 1/1] ONBUILD RUN groupadd testgroup                                                        0.3s
------                                                                                                            
 > [step-2 1/1] ONBUILD RUN groupadd testgroup:
0.290 groupadd: group 'testgroup' already exists
------
Dockerfile.multistep:2
--------------------
   1 |     FROM onbuild-base AS step-1
   2 | >>> FROM step-1 AS step-2
   3 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c groupadd testgroup" did not complete successfully: exit code: 9

Additional info

No response

[thaJeztah]

Possibly duplicate of;

• ONBUILD executed in multi-stage build grandchild moby/buildkit#5578

Which was fixed by

• dockerfile: fix onbuild propagation for child stages moby/buildkit#5593

(which is part of BuildKit v0.18.2)

[jfine]

Thanks @thaJeztah. I wasn't completely sure where best to file this issue. This does appear to be a duplicate of that.

[jfine]

@thaJeztah @tonistiigi do you happen to have an example of how to use 0.18.2 (https://github.com/moby/buildkit/releases/tag/v0.18.2) with docker build or how to add it to the builder instances for use?

[tonistiigi]

Add # syntax=docker/dockerfile:1.12 on top of the Dockerfile for this fix if your default BuildKit is not v0.18.2 yet.

[jfine]

@tonistiigi that worked perfect! Thanks!