runtime: some Go executables broken with macOS 10.12.4 / Xcode 8.3 but fine with Xcode 8.2

[dmaclach]

See couchbase/sync_gateway#2417 for repro steps.

System:

MacBook Pro (x86_64)
macOS 10.12.3 (Sierra)
go version go1.8 darwin/amd64
Xcode 8.3; clang -v
Apple LLVM version 8.1.0 (clang-802.0.38)
Target: x86_64-apple-darwin16.4.0
Thread model: posix

What did you do?

See description at
couchbase/sync_gateway#2417

I also found this with some code I am unable to share.

My code does work fine with Xcode 8.2:
clang -v
Apple LLVM version 8.0.0 (clang-800.0.42.1)
Target: x86_64-apple-darwin16.4.0
Thread model: posix

and from reading the bug linked above it works fine there as well.

[favadi]

I have the same problem as OP but only for some binaries, not all. When trying to start, it is killed instantly.

$ ./test
Killed: 9

I can't figure out the common pattern for broken programs.

[rasky]

Reproduction:

package main

import (
	"fmt"

	_ "github.com/veandco/go-sdl2/sdl"
)

func init() {
	fmt.Println("init")
}

func main() {
	fmt.Println("Hello, world")
}

 $ ./testx
 fish: './testx' terminated by signal SIGKILL (Forced quit)

Importing go-sdl2 is enough to make it crash, the prints are just to show that the runtime is unable to get to main.init. go-sdl2 has a single init() function, but commenting it out doesn't change anything, it still crashes right away.

I installed SDL itself via homebrew, the bottled version, a few weeks ago.

[rasky]

Basically, just using cgo and linking against sdl2 causes the crash:

package sdl

// #cgo linux freebsd darwin pkg-config: sdl2
import "C"

If you make the above file the whole go-sdl package, the binary will still crash.

[rasky]

Uhm it looks like lldb is even unable to spawn the binary:

(lldb) run
error: error: ::posix_spawnp ( pid => 24893, path = '/Users/rasky/Sources/go/src/ndsemu/testx/testx', file_actions = 0x7fff555225e8, attr = 0x7fff555225f8, argv = 0x7fd1e92a8150, envp = 0x7fd1e92ad520 ) err = Cannot allocate memory (0x0000000c)

so it seems like the binary is being linked in a way that the OS refuses to run it.

/cc @ianlancetaylor

[fighterlyt]

just the same, cgo make it crash

[crazytyper]

same here but with github.com/shirou/gopsutil/cpu instead of sdl2.

trying to run the binary with dtruss gives Unknown error.

[rgeronimi]

Same here, very simple to reproduce:

$ go test github.com/blevesearch/cld2
signal: killed
FAIL	github.com/blevesearch/cld2	0.003s

Investigating the MacOS system logs, it seems to be an issue with taskgated:
taskgated MacOS error: -67062
taskgated no signature for pid=4766 (cannot make code: UNIX[No such process])

Maybe a code signature issue around cgo...

[rasky]

I'll notice that all Go binaries (including working ones) generate the -67062 error, but I think that's normal because they are not codesigned. The errors I get only when running these crashing cgo binaries are:

default	14:32:47.099750 +0200	taskgated	UNIX error exception: 3
default	14:32:47.099853 +0200	taskgated	no signature for pid=28180 (cannot make code: UNIX[No such process])

[rgeronimi]

@rasky
On my environment (MacOS 10.12.14, Xcode8.3), non-cgo go programs do not generate -67062 errors. In my tests I performed, this error was strictly correlated to (1) crashes with the "signal: killed" message (2) cgo programs.
Additionally, I also get the "UNIX error exception: 3" message in the system logs

[margerum]

getting signal: killed when trying to use go run *.go

go build works, but then i get "Killed: 9" when I try to run the app

[aprilchild]

same here, dowloaded Xcode 8.2.1, replaced in /Applications and it works again..

[repustate]

For everyone running into this issue, I've been able to build binaries by adding -ldflags -s during the go build phase.

[ianlancetaylor]

By default on Darwin the linker will invoke dsymutil to put the debug info directly into the executable. This invocation of dsymutil is disabled by the linker's -s option. Perhaps this is really a bug in dsymutil, or in how dsymutil deals with files created by the Go linker. Could somebody with an affected system please try modifying the code in Link.hostlink in cmd/link/internal/ld/lib.go to skip invoking dsymutil, and see if that fixes the problem? Thanks.

[josharian]

@ianlancetaylor that does fix the problem. Tested on 1.8.

[ianlancetaylor]

The call to dsymutil was added to fix #8973. I suppose the next question is: if we don't call dsymutil, what happens when you try to debug a Go program built with cgo?

[cosnicolaou]

I'm seeing this without using cgo, what's the suggested work around - use xcode 8.2?

[bradfitz]

@cosnicolaou, see above (#19734 (comment)). Does that not work for you?

[cosnicolaou]

thanks, it works, from reading the comments I thought this was a cgo only problem/fix. It's also painful for our build/automation for obvious reasons.

[travissanderson-wf]

would adding these flags to build scripts have any negative repercussions that you're aware of? performance implications or anything like that?

[cosnicolaou]

no, it's just a pain to find all of the invocations etc.

[travissanderson-wf]

agreed :) thanks for verifying they are safe to add though

[rsc]

I have been assuming that takes effect with Go 1.8: Go 1.9 - the first release with that policy - won't invalidate Go 1.8; Go 1.10 will.

[bradfitz]

Well, as one datapoint, Kubernetes still hasn't moved from Go 1.7 to Go 1.8, so this means you can't run Kubernetes (even Kubernetes 1.6) on a Mac now.

[bradfitz]

Er, build, not run. Maybe that's okay.

[rsc]

Right, build, and even then only build with newer Xcode. It's not like the gettimeofday crashes.

[bradfitz]

This may impact their point release builds, but I'm not sure how Kubernetes does their releases (on an actual Mac, or cross-compiled).

Running file on https://storage.googleapis.com/kubernetes-release/release/v1.6.1/bin/darwin/amd64/kubectl just says kubectl: Mach-O 64-bit executable x86_64. (no ldd... I don't know Mac tools)

[rsc]

They can always add -ldflags=-s or update to Go 1.8.1 like everyone else. Also, kubectl at least doesn't appear to have any cgo code in it (otool -l kubectl | grep nundefsym).

[abh]

Yeah, kubectl seems to be cross compiled (it doesn't use the macOS DNS resolver).

[vyorkin]

none of the above workarounds worked for me :(

mac os x sierra 10.12.4
xcode 8.2 (8C38)
command line tools: http://adcdownload.apple.com/Developer_Tools/Command_Line_Tools_macOS_10.12_for_Xcode_8.2/Command_Line_Tools_macOS_10.12_for_Xcode_8.2.dmg

before installing command line tools I've removed /Library/Developer/CommandLineTools directory

UPD: it works now! after uninstalling the previous version with brew uninstall ethereum and installing a new one from the develop branch with brew install ethereum --devel

[pawelkowalak]

Update to Go 1.8.1, it is fixed now.

[rodrigo-brito]

Go 1.8.1 Works! 🎉

[OSMeteor]

use go1.8.1 fixed it good