Git submodule error when building a container through 'docker-compose up --build'

[nakulm95]

Description
Unable to build a service through the 'build' directive, providing an https link for the context.
For example:

  service1:
    build:
      context: https://<link_to_service.git>
      dockerfile: Dockerfile

build process is invoked by the following command:

docker-compose up --build -d

Steps to reproduce the issue:

1. On a system with git 1.8.3.1 & docker-compose 2.5.1, use the structure above to build a container, providing an https link for the context

Describe the results you received:
Build process fails with the following error:
#3 [servicesconfig_service internal] load git source https://<path_to_service.git>
#0 0.224 1b253bb1ab8bacc2be8bc0d765e34184bc94eb68 refs/heads/master
#0 0.423 usage: git submodule [--quiet] add [-b ] [-f|--force] [--name ] [--reference ] [--] []
#0 0.423 or: git submodule [--quiet] status [--cached] [--recursive] [--] [...]
#0 0.423 or: git submodule [--quiet] init [--] [...]
#0 0.423 or: git submodule [--quiet] deinit [-f|--force] [--] ...
#0 0.423 or: git submodule [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...]
#0 0.423 or: git submodule [--quiet] summary [--cached|--files] [--summary-limit ] [commit] [--] [...]
#0 0.423 or: git submodule [--quiet] foreach [--recursive]
#0 0.423 or: git submodule [--quiet] sync [--recursive] [--] [...]
#3 ERROR: failed to update submodules for https://<path_to_service.git>: exit status 1

Describe the results you expected:
Expected the build process to succeed since there's no public documentation on git version requirements with docker-compose.

Additional information you deem important (e.g. issue happens only occasionally):
This issue is happening consistently in our jenkins environment.

Output of docker compose version:

+ docker-compose version
Docker Compose version v2.5.1

Output of docker info:

+ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.7.1-docker)
  scan: Docker Scan (Docker Inc., v0.12.0)

Server:
 Containers: 41
  Running: 0
  Paused: 0
  Stopped: 41
 Images: 837
 Server Version: 20.10.12
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7b11cfaabd73bb80907dd23182b9347b4245eb5d
 runc version: v1.0.2-0-g52b36a2
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.0-1160.45.1.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 15.51GiB
 Name: <redacted>
 ID: <redacted>
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details:

+ git version
git version 1.8.3.1

The dockerfile in question doesn't have any git submodule instructions and works on my macbook (running git v2.33.0, docker-compose v2.5.1)

[maxcleme]

@nakulm95

I've tried to replicate the issue on my end with dummy a repo using submodule, and everything works as expected here.

May I asked for more information regarding that repo or the Dockerfile it contains ?

Are you able to create a dummy example to replicate the issue without any sensitive information ?

[ndeloof]

Please try to build directly using buildx (which is used under the hood by compose) : docker buildx build https://<link_to_service.git>. If this one suffers from the same issue, please report but on github.com/docker/buildx

[laurazard]

Closing since there's been no activity for a while, please comment/ping with more information for us to reopen it (like what @ndeloof said).

[eruizc-dev]

I'd like to reopen this issue, as I'm currently having the same problem while providing an example you can try:

Basically when building Ollama I get the following error:

3.310 + git submodule init
3.321 fatal: not a git repository (or any of the parent directories): .git
3.321 llm/llama.cpp/generate_linux.go:3: running "bash": exit status 128

Here's (a simplified) docker-compose.yml

version: '3'

services:
  ollama:
    entrypoint: /go/src/github.com/jmorganca/ollama/ollama
    command: serve
    build:
      context: https://github.com/jmorganca/ollama.git#main
      dockerfile: Dockerfile.build
      args:
        GOFLAGS: "'-ldflags=-w -s -X=github.com/jmorganca/ollama/server.mode=release'"
    restart: unless-stopped

And here's an equivalent buildx command that exits with the same error

docker buildx build https://github.com/jmorganca/ollama.git\#main -f Dockerfile.build --build-arg=GOFLAGS="'-ldflags=-w -s -X=github.com/jmorganca/ollama/server.mode=release'"

---

Worth mentioning that if I clone the repository (without its submodules) and change the context accordingly, I can compile it no problems.

version: '3'

services:
  ollama:
    entrypoint: /go/src/github.com/jmorganca/ollama/ollama
    command: serve
    build:
      context: ./path/to/source
      dockerfile: Dockerfile.build
      args:
        GOFLAGS: "'-ldflags=-w -s -X=github.com/jmorganca/ollama/server.mode=release'"
    restart: unless-stopped

[glours]

Hi @eruizc-dev
Compose is using buildx to build the images, so if you're able to reproduce the issue with buildx directly the best option is to open an issue directly there

[eruizc-dev]

Thanks!

For anyone interested in this in the future, I'll be following this issue here: docker/buildx#2172