cmd/go: go-import metadata can't indicate a specific subdirectory within the repository

[owenhaynes]

What version of Go are you using (go version)?

$ go version
go version go1.12.7 linux/amd64

Does this issue reproduce with the latest release?

no ,but 1.12.7

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
OARCH="amd64"
GOBIN=""
GOCACHE="/workspace/.gocache"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/workspace/_go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build608236438=/tmp/go-build -gno-record-gcc-switches"

What did you do?

have module A in a repository call Foo
foo/
src/go/foo.go
src/go/go.mod
src/go/go.sum
foo.proto
foo.cpp
foo.h
Makefile

go mod file
module go.foo.app/foo

this is then masked behind a vanity URL
go.foo.app/foo => ssh://[email protected]/foo/foo.git

Use another module
module go.foo.app/test
go.foo.app/foo master

What did you expect to see?

Expected the module to be checked out into the the mod cache with the correct contents of src/go
Expected cache folder:
foo.go
go.mod
go.sum

What did you see instead?

the cache has the following
foo.proto
foo.cpp
foo.h
Makefile

It also remove the src/go folder, so something is working but in the wrong way.

With the introduction of modules was expecting it to be easier to contain go code in hybrid language repositories.

Also tried a vanity url of
go.foo.app/foo => ssh://[email protected]/foo/foo.git/src/go
but same affect.

[bcmills]

What go-import tag is the go.foo.app HTTP server returning for the repository?

Per https://tip.golang.org/cmd/go/#hdr-Remote_import_paths, for the configuration you describe to work, it should include a tag of the form

<meta name="go-import" content="go.foo.app git ssh://[email protected]/foo/foo.git">

where the prefix at the beginning of the content attribute matches the path at the root of the repository.

[bcmills]

Hmm, looking more closely at the configuration you describe, there is currently no way for the HTTPS server to indicate that the module is offset by the prefix src/go, since that is not a part of the Go package path. This layout is just not going to work with the go command's built-in git support.

You can either restructure the repo (or import path) so that the subdirectory corresponds to a suffix of the module path, or have the server extract the contents from the git repository itself and serve them using the module proxy protocol instead of relying on the go command's built-in git support.

[bcmills]

@owenhaynes, just to set expectations: I think it's unlikely that we would expand the go-import metadata format to support this use-case. If you can change either your import path or your repo structure to work with the format as it exists today, that's by far the simplest path forward.

[owenhaynes]

@bcmills We are planning on using a Athens once the authentication of the proxy has been sorted out, this was more just enquiring if the behaviour was correct of go get with modules and if it was possible to do without a module proxy.

I don't think we can make go get reject my repo as well because of backwards compatibility with none go modules. Its also strange that it removes files from the repo as well.

[bcmills]

Its also strange that it removes files from the repo as well.

That's because a repository can contain more than one module, either due to path suffixes or due to replace directives within the repo: a go.mod file normally indicates the boundary between two modules stored in the repo. Our extraction logic strips out those nested modules because they are not part of the requested module.

(In your case, since the nested go.mod file specifies module go.foo.app/foo, it could indicate the boundary between the go.foo.app/foo module itself and a copy of that module intended for use with a replace directive at some point within your development process.)

[bcmills]

this was more just enquiring if the behaviour was correct of go get with modules and if it was possible to do without a module proxy.

Yeah, the answer to that is that it looks like go get is working as designed here.

Based on that, I'm going to close out this issue, but please feel free to reopen or file a new issue if you run into further trouble.