cmd/go: document module-mode behavior of multi-element GOPATHs

[mvdan]

go version devel +311d87dbeb Thu Nov 29 08:30:13 2018 +0000 linux/amd64

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/mvdan/go/cache"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/mvdan/go/land:/home/mvdan/go"
GOPROXY=""
GORACE=""
GOROOT="/home/mvdan/tip"
GOTMPDIR=""
GOTOOLDIR="/home/mvdan/tip/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build190148469=/tmp/go-build -gno-record-gcc-switches"

In my case, mvdan.cc/sh/cmd/gosh is a main package inside the second GOPATH directory. It's also a module, so it can be built in either mode.

$ cd ~/go/src/mvdan.cc/sh/cmd/gosh
$ GO111MODULE=on go list -f {{.Target}}
/home/mvdan/go/land/bin/gosh
$ GO111MODULE=off go list -f {{.Target}}
/home/mvdan/go/bin/gosh

I understand why the GOPATH build puts the binary where it does - that's the old and documented logic.

And I too sort of understand why the module build goes into the first GOPATH entry - it's not building the package within any one GOPATH, so it just picks the first.

However, this can be very confusing. In my case, I ended up with two binaries, and because of PATH I was always running the older. The only mention is brief, under go help modules:

and installed commands (in GOPATH/bin, unless GOBIN is set)

I realise now that my fix should be to either set up a global GOBIN, or to stop using a multi-element GOPATH. I still think the documentation could be clearer, though. I think we should discourage the use of a multi-element GOPATH with no GOBIN in the modules world, because of the tricky scenario above.

/cc @bcmills @myitcv @rogpeppe

[bcmills]

I think we should discourage the use of a multi-element GOPATH with no GOBIN in the modules world, because of the tricky scenario above.

Having a multi-element GOPATH in the first place puts you firmly in the power-user category. I'm not sure that we need to warn about this because I'm skeptical that many users are in that situation to begin with...

[mvdan]

Agreed; I wonder if anyone else would even run into this issue. But still, it had me confused for a good twenty minutes, because I wasn't realising that I had two separate binaries for the same main package.

Perhaps a better generic warning would be on the use of a multi-element GOPATH. That is, "if you want to use this, you likely want to set GOBIN too". Or even "this likely doesn't make sense if you're going to work with modules".

[rogpeppe]

Having a multi-element GOPATH in the first place puts you firmly in the power-user category. I'm not sure that we need to warn about this because I'm skeptical that many users are in that situation to begin with...

I think that at least the "GOPATH and Modules" section in go help gopath should mention the expected behaviour when GOPATH has multiple elements (it always uses the first element for everything, presumably).

[perillo]

I also use a multi-element GOPATH, set to /home/manlio/.local/lib/go:/home/manlio/src/go and I have explicitly set GOBIN.

However I find it strange to have a GOPATH entry inside the other GOPATH entry. It should probably be disallowed. What is the reason for this choice?

[mvdan]

What is the reason for this choice?

The choice was arbitrary, and it doesn't really matter anymore :) I dropped that configuration over a year ago when I fully switched all my projects to modules.

[perillo]

I still have the multi-element GOPATH. Originally I used it to keep separate external packages and my packages, but now I'm still using it since it is nice to have the module cache in ~/.local.

[bcmills]

@perillo, note that it will likely be possible to configure the module cache location without reference to GOPATH in Go 1.15 (see #34527).

[perillo]

@bcmills, thanks. But what will happen to GOPATH? I hope it will not be removed since all my modules are inside GOPATH, with the property that

filepath.Join($GOPATH, module.Path) == module.Dir

I'm starting to call modules with this property local modules and I'm writing some tools to make it easy (as with gopath mode) to work with them.

[bcmills]

We're still figuring out the future of GOPATH, but nothing will fundamentally stop you from continuing to structure your local directory layout that way going forward.

(Just be aware that modules, unlike GOPATH directories, can interleave: package example.com/foo/bar may be provided by module example.com, even if package example.com/foo is in its own distinct module in the middle.)

[eddycjy]

@bcmills I recently read the latest official blog post about its plan to remove GOPATH in Go1.17. Is this too unreasonable for a historical project?

[matloob]

At this point, I wonder whether we should instead update go help gopath to introduce GOPATH in module mode as holding a single path. We could then continue with documenting GOPATH in the legacy GOPATH mode as holding multiple values.

I'm not sure this is a problem in practice though.