Docker build cannot resolve git context with html escapes

[JackSullivan]

Description

Running docker build when the context is a git repository with html escaped characters fails with an error message that seems to be related to the characters becoming unescaped.
Steps to reproduce the issue:
Concretely, for a remote git repository located at git://foo%[email protected]:2222/myrepo.git

1. docker build -f Dockerfile git://foo%[email protected]:2222/myrepo.git

Describe the results you received:
Gives unable to prepare context: unable to 'git clone' to temporary context directory: error fetching: fatal: unable to look up [email protected]@gitrepos.barcorp.com (port 2222) (Name or service not known)

It seems, based on the error message, that the git url is being (incorrectly) being html unescaped.

Describe the results you expected:
Successfully pulling in the git repo as a context for the Dockerfile

Output of docker version:

Client:
 Version:           18.09.1-ol
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        e32a1bd
 Built:             Thu Jun  6 14:55:42 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.1-ol
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       e32a1bd
  Built:            Thu Jun  6 14:47:41 2019
  OS/Arch:          linux/amd64
  Experimental:     false
  Default Registry: docker.io

Output of docker info:

Containers: 9
 Running: 0
 Paused: 0
 Stopped: 9
Images: 19
Server Version: 18.09.1-ol
Storage Driver: overlay2
 Backing Filesystem: xfs
 Supports d_type: true
 Native Overlay Diff: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version:
runc version: 4bb1fe4ace1a32d3676bb98f5d3b6a4e32bf6c58
init version: fec3683
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.14.35-1902.4.8.el7uek.x86_64
Operating System: Oracle Linux Server 7.7
OSType: linux
Architecture: x86_64
CPUs: 88
Total Memory: 424.8GiB
Name: jaqen.us.oracle.com
ID: CPCM:K56X:HVRK:JUOQ:DRHM:7DAO:45CX:JID6:D2YQ:ZIFJ:5RZH:SIQI
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
HTTP Proxy: http://www-proxy-ash7.us.oracle.com:80/
HTTPS Proxy: http://www-proxy-ash7.us.oracle.com:80/
No Proxy: localhost,.oraclecorp.com,.us.oracle.com
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

[thaJeztah]

This error is coming directly from git. When building from a remote URL, the CLI uses git to clone the repository in a temp-dir, after which it is used to perform the build;
https://github.com/docker/cli/blob/v19.03.4/vendor/github.com/docker/docker/builder/remotecontext/git/gitutils.go#L202

If you try the steps that the CLI does in a shell, you'll see the same error;

mkdir repro-2164 && cd repro-2164
git init
# Initialized empty Git repository in /Users/sebastiaan/triage/repro-2164/.git/

git remote add origin 'git://foo%[email protected]:2222/myrepo.git'
git fetch --depth 1 origin -- master
# fatal: unable to look up [email protected] (port 2222) (nodename nor servname provided, or not known)

Looking at the example you provided, I suspect you used the wrong HTML code; %04 is not a regular character, but a control character (ASCII "EOT" (end of transmission))

If you meant to encode an @, symbol, you'd be looking for %40; using that, you'll get a more reasonable error (and this would likely work in your case, with the right username and URL):

docker build 'git://foo%[email protected]:2222/myrepo.git'
# unable to prepare context: unable to 'git clone' to temporary context directory: error fetching: fatal: unable to look up [email protected]@gitrepos.barcorp.com (port 2222) (Name does not resolve): exit status 128

I'll close this ticket, because I don't think there's a bug, but feel free to continue the conversation

[JackSullivan]

I mistyped in the original issue, so %04 was supposed to be %40, the url encoded version of @. The error message that comes back is the same as that in the original issue above, where the response has the unencoded symbol (ie. foo%40barcorp.com@gitrepos in the input becomes [email protected]@gitrepos in the error message.

However, after doing some more debugging based on the commands that you posted, it seems that my private repository does not support the git:// protocol. I've tried a number of ssh variations, but none of them get detected as a url, and it seems be due to using a non default port.

Concretely, I've tried:

• docker build -f Dockerfile ssh://foo%[email protected]:2222/myrepo.git
• docker build -f Dockerfile foo%[email protected]:2222/myrepo.git
• docker build -f Dockerfile foo%[email protected]:2222:myrepo.git
  Which all give the same error:
  unable to prepare context: path ... not found
  Which seems to me to indicate that docker is treating all of these as local paths rather than git repositories.

How does one connect to a remote git repo through ssh with a non-standard port?

[JackSullivan]

@thaJeztah Is there a path for connecting to non-standard ports through ssh as I alluded to in the above comment?

[thaJeztah]

I think the problem in that case is that the helper-package used in Docker currently doesn't support ssh: as protocol, thus won't try to use it as an external URL, but as a local filepath; https://github.com/moby/moby/blob/9419024554451ba4766766672f934d1ba14834d8/builder/remotecontext/git/gitutils.go#L205-L209

Should probably be possible to add, but given that that's an enhancement/new feature, that likely won't qualify as a backport for existing releases