cmd/go/internal/envcmd: GOCACHE=off behavior difficult to debug

[kevinburke]

Please answer these questions before submitting your issue. Thanks!

What did you do?

I have a test that calls go build with the following code:

	env := []string{
		"GOROOT=" + os.Getenv("GOROOT"),
		"GOPATH=" + filepath.Join(root, "src", "tool", "_gopath"),
	}
        cmdArgs := []string{"build", "-o", binPath}
	cmd := exec.Command("go", cmdArgs...)
	cmd.Env = append(env, "GOOS="+p.OS.String(), "GOARCH="+p.Arch.String())

Compiling with tip this failed first with the error "exit status 1", then when I printed out the stdout/stderr I got:

build cache is disabled by GOCACHE=off, but required as of Go 1.12

This was very confusing because I double and triple checked the env vars and the source code and I never referred to GOCACHE.

Finally, by editing the cmd/go source code and recompiling I was able to determine the actual problem was that I was not passing through the $HOME variable, so os.UserCacheDir() was failing, and cmd/go/internal/cache/default.go:DefaultDir was returning an error. $HOME is not a requirement to use the build cache on Go 1.11, or, perhaps this test program was failing to hit the build cache in Go 1.11 and it was not an error.

This is concerning because we should not expect the average Go user to recompile the Go command to determine what is wrong with their program.

It would be good to provide an error message that lets the user better debug what is wrong with their program and how/why GOCACHE is being set to off in the build environment.

Does this issue reproduce with the latest release (go1.11.2)?

No

System details

go version devel +a6293995c5 Thu Dec 13 23:58:06 2018 +0000 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/kevin/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/kevin"
GOPROXY=""
GORACE=""
GOROOT="/Users/kevin/go"
GOTMPDIR=""
GOTOOLDIR="/Users/kevin/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
GOROOT/bin/go version: cache default dir /Users/kevin/Library/Caches/go-build
go version devel +a6293995c5 Thu Dec 13 23:58:06 2018 +0000 darwin/amd64
GOROOT/bin/go tool compile -V: cache default dir /Users/kevin/Library/Caches/go-build
compile version devel +a6293995c5 Thu Dec 13 23:58:06 2018 +0000
uname -v: Darwin Kernel Version 18.2.0: Fri Oct  5 19:41:49 PDT 2018; root:xnu-4903.221.2~2/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.14.1
BuildVersion:	18B75
lldb --version: lldb-1000.0.38.2
  Swift-4.2

[kevinburke]

see also #29251, it's concerning that this error message has appeared twice, which increases the likelihood (to me anyway) that a significant portion of the user base will hit it.

[bcmills]

$HOME is not a requirement to use the build cache on Go 1.11

On Darwin, the build cache requires either GOCACHE or os.UserCacheDir(), and os.UserCacheDir() requires HOME.

or, perhaps this test program was failing to hit the build cache in Go 1.11 and it was not an error.

That seems more likely. But we should certainly fix the diagnostics.

[bcmills]

FWIW, the diagnostic in this case is intended to read build cache is required, but could not be located: $HOME is not defined.

The logic for that is here, and seemed so trivial that I only tested one mode:

go/src/cmd/go/internal/cache/default.go

Lines 74 to 83 in 47fb1fb

	// die calls base.Fatalf with a message explaining why DefaultDir was "off".
	func die() {
	if os.Getenv("GOCACHE") == "off" {
	base.Fatalf("build cache is disabled by GOCACHE=off, but required as of Go 1.12")
	}
	if _, err := os.UserCacheDir(); err != nil {
	base.Fatalf("build cache is required, but could not be located: %v", err)
	}
	panic(fmt.Sprintf("cache.die called unexpectedly with cache.DefaultDir() = %s", DefaultDir()))
	}

Obviously I should have tested more thoroughly. 🙂

[FiloSottile]

Are we committed to not letting users build code unless they set $HOME or $GOCACHE? I'd expect this to break quite a few Docker images, which probably didn't want to keep the cache around anyway. Also, I run some machines with a read-only $HOME, which I guess won't be able to build Go code anymore.

It already broke tip.golang.org and stuff inside Google, too.

[bcmills]

Are we committed to not letting users build code unless they set $HOME or $GOCACHE?

It's what we announced in the 1.11 release notes, and nobody complained about it then.

We could perhaps put the cache in os.TempDir() if we can't find it some other place, though.

[bcmills]

At any rate, that seems like a separate issue from the broken diagnostic message.

[kevinburke]

Apologies for not complaining, I guess I didn’t realize the implications. I compile from tip daily and just noticed it now

On Fri, Dec 14, 2018 at 18:44 Bryan C. Mills ***@***.***> wrote: At any rate, that seems like a separate issue from the broken diagnostic message. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#29243 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAOSI9PaxaLmLPhn3XD-j4eg_CsdVHKeks5u4_F3gaJpZM4ZSsld> .

-- Kevin Burke phone: 925.271.7005 | kev.inburke.com

[FiloSottile]

At any rate, that seems like a separate issue from the broken diagnostic message.

Opened #29267.

[gopherbot]

Change https://golang.org/cl/154309 mentions this issue: cmd/go/internal/cache: save more data from DefaultDir

[purpleidea]

build cache is disabled by GOCACHE=off, but required as of Go 1.12

Just started seeing this in all my travis builds. Which broke everything! I set this variable in my .envrc because I NEVER want the tests cached! How else can I disable test caching now??

[ianlancetaylor]

You can always disable test caching by using the -test.count=1 option, as documented at https://golang.org/cmd/go/#hdr-Test_packages.

[gopherbot]

Change https://golang.org/cl/163798 mentions this issue: playground: use Go 1.12