[BUG] service can't be used with extends as it declare depends_on

[r3nor]

Description

When trying to bring up a compose project that makes use of extends feature, I am getting the following error:

service "nginx" can't be used with `extends` as it declare `depends_on`

Steps To Reproduce

1. Have a docker-compose.yml project with extends, like this:

nginx:
    extends:
      file: ./nginx/compose.yml
      service: nginx
    container_name: "${SERVER_NAME:}-nginx"
    env_file: .env

and the nginx/compose.yml:

version: "3.9"

services:
  nginx:
    image: nginx:1
    restart: unless-stopped
    depends_on:
      - web_app
    volumes:
      - ./html:/var/www/html

2. docker compose up

3. See error: service "nginx" can't be used with extends as it declare depends_on

Compose Version

2.24.6

Docker Environment

N/A

Anything else?

No response

[ndeloof]

This is the expected behavior, see https://github.com/compose-spec/compose-spec/blob/master/05-services.md#restrictions

[meriouma]

This seems to be a new restriction added in 2.24.6. We have an extends combined with depends_on and everything was working fine with 2.24.5 and previous versions. Not sure why this wasn't flagged as a breaking change?

[r3nor]

Yeah, same for me. Was working fine on 2.24.5 and broke on this new 2.24.6 release.

[meriouma]

This makes reusing blocks much harder, as fragments cannot be used across files. As far as I know, using extends was the only way to have reusable depends_on blocks while allowing to separate services in multiple files and have a common file to extend from.

[ndeloof]

restriction already was implemented in 2.23.x, new compose parser used in 2.24.x prior to latest release was just missing this check.

[r3nor]

@meriouma maybe you want to check out include: https://docs.docker.com/compose/compose-file/14-include/

[nhorton]

@meriouma maybe you want to check out include: https://docs.docker.com/compose/compose-file/14-include/

That is not sufficient. It is common to do stuff like have an image like "background worker" and then have 3 instances of it - high-pri, medium-pri, and low-pri. You could then have the base depend on stuff like your DB and message bus and such, then just have 3 instances that extend the base. Now you have to repeat those dependencies over and over in the instantiations. This is definitely more degradation in compose.

We have used Compose for development purposes with a Kubernetes cluster for deployment. It deeply feels like Compose has been losing functionality while Kubernetes has been gaining it, and Kind is becoming more and more compelling to replace Compose from these sorts of things.

[davidlougheed]

encountering this too. why is a breaking change being included in a patch release at all?

[davidlougheed]

what's also frustrating here is that everything used to work as-intended before compose 2.24 and its use of a not-even-prod-ready compose-go project, but there wasn't really a clear alert that everything was going to break.

[ndeloof]

@davidlougheed compose-go/v2 is not "not-even-prod-ready". We released as RC as we anticipated the need for some additional refactoring, and don't want third party library to suffer from API changes.

THAT BEING SAID: as compose-go includes a consistency check, it would be able to detect reference to another service is orphaned after being used by extends, such a check docker compose v1 didn't implemented (so this rule in the compose file format, inherited by the compose specification). With the updated architecture of compose-go/v2 we can now relax this constraint, and offer more flexibility to users (which still should read the docs before they complain about a feature ). I'll write a proposal in the compose-spec repo for this purpose, so we can get this both fixed in the docs and in the code

[Leonid99]

This is the expected behavior, see https://github.com/compose-spec/compose-spec/blob/master/05-services.md#restrictions

@ndeloof So, what is the supported way of extending services that depend on other services?

[ndeloof]

@Leonid99 there's no way currently

[luciangabor]

It's possible to merge depends_on "later":

$ find *.yaml -print -exec cat {} \;
compose.yaml
include:
  - path:
      - defs.yaml
      - deps.yaml
defs.yaml
services:
  x:
    image: x

  y:
    image: y
#    depends_on:
#      - x

  z:
   extends: y
deps.yaml
services:
  y:
    depends_on:
      - x

$ docker compose config
name: deps
services:
  x:
    image: x
    networks:
      default: null
  "y":
    depends_on:
      x:
        condition: service_started
        required: true
    image: "y"
    networks:
      default: null
  z:
    image: "y"
    networks:
      default: null
networks:
  default:
    name: deps_default

[luciangabor]

... and if you want to "inherit" the depends_on, deps.yaml could be like this:

services:
  y: &idem
    depends_on:
      - x
  z: *idem

[pironim]

Got this error with ubuntu update

from docker-compose-plugin (2.24.6-1ubuntu.22.04jammy)
to (2.24.5-1ubuntu.22.04jammy)

[luciangabor]

It's quite straightforward to see why is was not supposed to work if you consider such a case:

With a compose.yaml like this:

services:
  service:
    extends:
      file: file.yaml
      service: service

that extends a service defined in a file.yaml like this:

services:
  hidden:
    image: hidden

  service:
    image: image
    depends_on:
      - hidden

[ndeloof]

sure, but the same applies to other resources, without such a constraint:

file.yaml

services:
  service:
    image: image
    volumes:
      - hidden:/work

volumes:
  hidden: ...

[luciangabor]

about that, extending services defined like this: https://docs.docker.com/compose/compose-file/05-services/#ipv4_address-ipv6_address doesn't work either, but it's not caught by checks if there's a homonym network around

[ndeloof]

that's my point: consistency check just validate there's a resource for any declared reference, whenever this might not have been declared in the original compose file. As we already do for networks/volumes, I don't see any reason we don't do the same for dependent services

[Leonid99]

The problem with this approach is that if more than one service derives from the base service (and this is always the case — why would you go into the effort of sharing functionality if there is just one service), you will need to copy-paste the “depends_on” stanza into each one. Also, if there is more than one service in the derivation chain, it will make the derived services unusable on their own, i.e. consider a situation where you have service “x" extending service “y" which extends service “z”, all of which depend on the service “foo”. This change moves the “depends_on foo” logic from z to x. Before this change, you could happily start a container with services z and y, while after the change, only service x is usable.

…

On Feb 28, 2024 at 4:41:39 AM, luciangabor ***@***.***> wrote: It's possible to merge depends_on "later": $ find *.yaml -print -exec cat {} \;compose.yamlinclude: - path: - defs.yaml - deps.yamldefs.yamlservices: x: image: x y: image: y # depends_on: # - x z: extends: ydeps.yamlservices: y: depends_on: - x $ docker compose configname: depsservices: x: image: x networks: default: null "y": depends_on: x: condition: service_started required: true image: "y" networks: default: null z: image: "y" networks: default: nullnetworks: default: name: deps_default — Reply to this email directly, view it on GitHub <#11544 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFYDJ2SWSFO7HK2ERSK3VDYV33VHAVCNFSM6AAAAABDZYY2JKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRYGU4TCNRRG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[davidlougheed]

hmm I removed my extends in favour of includes + merging and now I get bizarre network errors where services are not joining some (but not all) of the declared networks, leading to name resolution errors. maybe related, or maybe not. curious if anyone else encounters this.

Edit: ah no it's unrelated, I'm also encountering a different compose-go bug: #11533

[faelin]

Now that the spec has been updated, is there a projected timeline on updating the compose project to meet the revised spec?

[ndeloof]

will be part of tomorrow's release

[faelin]

Thanks for the fast update @ndeloof!

[ctrlaltf24]

@r3nor Try now, latest published version fixed it for me (2.25.0).

$ docker compose version
Docker Compose version v2.25.0
$ docker --version
Docker version 26.0.0, build 2ae903e

Thanks @ndeloof !

[sheyko94]

Hello there, do we have an ETA for the version 2.25 to be released on Docker Desktop? My company heavily depends on this fix, for now I workaround the issue by download docker-compose instead and working with it, but the entire company is trained on the platform to use docker compose - would be nice to have the fix in place.

[matthiasPOE]

@ndeloof does this mean we should stop extending services as this feature will be deprecated in future?

[ndeloof]

@matthiasPOE no, we relaxed this restriction in the spec compose-spec/compose-spec#470
issue is closed by release v2.26.0

[gander]

Hello there, do we have an ETA for the version 2.25 to be released on Docker Desktop? My company heavily depends on this fix, for now I workaround the issue by download docker-compose instead and working with it, but the entire company is trained on the platform to use docker compose - would be nice to have the fix in place.

I was looking for the directory where this plugin is located to replace it manually. After several unsuccessful attempts, I replaced docker-compose in all found directories and one of them worked. I don't know which one, but you can also try it yourself.

https://github.com/docker/compose#linux

---

Don't rely on the about page, it still shows the old version:

docker compose version
# Docker Compose version v2.26.0

[gander]

/usr/lib/docker/cli-plugins/docker-compose

[Morriz]

@gander are you referring to OSX Docker Desktop? That location does not exist in my version of Sonoma.

[gander]

@Morriz

@gander are you referring to OSX Docker Desktop? That location does not exist in my version of Sonoma.

No, my Linux Mint location. Check other from https://github.com/docker/compose or use find or command to locate your docker-compose

[Morriz]

Yeah, tnx. I was able to follow the original install instructions from Docker.

[beni0888]

Any idea about when the fix will be released in OSX Docker Desktop?

[ndeloof]

should happen in a few days

[beni0888]

In case someone else is facing issues because Docker Desktop is still running an old version of the docker-compose plugin, I've found this useful video explaining how to easily upgrade the docker-compose plugin used by Docker Desktop.

[emersonfoni]

@beni0888 you didn't link the video 🙂

[beni0888]

You're right! Fixed!!