cmd/compile: 'internal compiler error: bvbulkalloc too big' when compiling a file containing a large map

[mcdee]

$ go version
go version devel +2d6ee6e89a Thu Aug 1 20:37:08 2019 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jgm/.cache/go-build"
GOENV="/home/jgm/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/jgm/.go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/jgm/snippets/goroot"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/jgm/snippets/goroot/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build261965408=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Attempting to go build on a file that creates a large map fails:

$ go build
# _/home/jgm/src/go/buildfail
./signatures.go:146811:24: internal compiler error: bvbulkalloc too big: nbit=5752 count=3948889 nword=180 size=710800020

goroutine 22 [running]:
runtime/debug.Stack(0xfdd400, 0xc00000e018, 0x0)
        /home/jgm/snippets/goroot/src/runtime/debug/stack.go:24 +0x9d
cmd/compile/internal/gc.Fatalf(0xe54748, 0x36, 0xc3c222d7f8, 0x4, 0x4)
        /home/jgm/snippets/goroot/src/cmd/compile/internal/gc/subr.go:188 +0x291
cmd/compile/internal/gc.bvbulkalloc(0x3c415900001678, 0x89ba0, 0x89ba0, 0xc8e9cd8000, 0xc3c222d8b8)
        /home/jgm/snippets/goroot/src/cmd/compile/internal/gc/bv.go:34 +0x1c4
cmd/compile/internal/gc.newliveness(0xc00029e160, 0xc00029e420, 0xc0cb6c8000, 0x1678, 0x1c00, 0xc07a7bf560, 0xb3e0, 0xe3e6d1)
        /home/jgm/snippets/goroot/src/cmd/compile/internal/gc/plive.go:503 +0x199
cmd/compile/internal/gc.liveness(0xc02f747e60, 0xc00029e420, 0xc0a752f0a0, 0x0, 0xe3e6de, 0xd)
        /home/jgm/snippets/goroot/src/cmd/compile/internal/gc/plive.go:1394 +0x95
cmd/compile/internal/gc.genssa(0xc00029e420, 0xc0a752f0a0)
        /home/jgm/snippets/goroot/src/cmd/compile/internal/gc/ssa.go:5286 +0x92
cmd/compile/internal/gc.compileSSA(0xc00029e160, 0x3)
        /home/jgm/snippets/goroot/src/cmd/compile/internal/gc/pgen.go:308 +0x3c2
cmd/compile/internal/gc.compileFunctions.func2(0xc030fb5ec0, 0xc0072d7e10, 0x3)
        /home/jgm/snippets/goroot/src/cmd/compile/internal/gc/pgen.go:363 +0x49
created by cmd/compile/internal/gc.compileFunctions
        /home/jgm/snippets/goroot/src/cmd/compile/internal/gc/pgen.go:361 +0x128

This contains approximately 140K additions, carried out as individual calls (rather than initialising the map during declaration). Relevant parts of code are:

  type function struct {
      name   string
      params []string
  }
  
  var functions map[uint32]function
  
  func InitFunctionMap() {
      functions = make(map[uint32]function)
      functions[305651098] = function{name: "decimalMul", params: []string{"uint256", "uint256"}}
      functions[3393457315] = function{name: "decimalDiv", params: []string{"uint256", "uint256"}}
      ...

A full copy of the file is at https://github.com/wealdtech/compilebug

What did you expect to see?

Would expect the build to complete.

What did you see instead?

Error output as above after approximately 50 minutes of building.

[bcmills]

Does this also reproduce using Go 1.12.7, or is it a regression in 1.13?

CC @randall77 @griesemer

[mcdee]

Yes, same issue with 1.12.7

[ALTree]

AFAIK this is not new. See #26560 (comment). It can be triggered by huge, auto-generated literals or functions (I've triggered this a few times while fuzzing the compiler with really big autogenerated functions).

The standard workaround is to copy the data in the map in init().

[ALTree]

Clearly it still would be interesting to know if there was a regression on this, i.e. if a map of the same size compiled fine in Go1.10 and it doesn't now.

[mcdee]

@ALTree I'm not sure what you mean by "copy the data in the map in init()" here. As per the original report, the map is created and individual items added on an entry-by-entry basis rather than initialising the map with all entries in one go.

[ALTree]

@mcdee

I used a smaller version of your reproducer with ~5000 map entries. Your code does this:

var functions map[uint32]function

func main() {
	functions = make(map[uint32]function)
	functions[305651098] = function{name: "decimalMul", params: []string{"uint256", "uint256"}}
	functions[3393457315] = function{name: "decimalDiv", params: []string{"uint256", "uint256"}}
        // other 5000 entries
}

Compare with this:

var functions map[uint32]function

var functionsKeys = []uint32{
	305651098,
	3393457315,
        // other 5000 keys
}

var functionsValues = []function{
	function{name: "decimalMul", params: []string{"uint256", "uint256"}},
	function{name: "decimalDiv", params: []string{"uint256", "uint256"}},
        // other 5000 values
}

func init() {
	functions = make(map[uint32]function)
	for i := 0; i < len(functionsKeys); i++ {
		functions[functionsKeys[i]] = functionsValues[i]
	}
}

Note how the second version initializes the map in init().

Compiling the first version:

$ time go tool compile signatures.go 

real	0m5.687s
user	0m12.209s
sys	0m0.167s

Compiling the second version:

$ time go tool compile signatures2.go 

real	0m0.157s
user	0m0.247s
sys	0m0.020s

[mcdee]

@ALTree thanks. This does work, but increases the size of my binary by about 18MB compared to the prior runtime initialisation method I used e.g. rather than functions[305651098] = function{name: "decimalMul", params: []string{"uint256", "uint256"}} I used addFunction("decimalMul(uint256,uint256)") where addFunction() parsed the string to break out the relevant values and add the map entry.

Looks like I'll take the runtime parsing hit and stick with my existing method. Should this be left open for someone to address the issue of the compiler failure?

[randall77]

Yes, you can leave this open. The complete example will help.

[odeke-em]

@randall77, @mcdee posted a full example which you can access at https://raw.githubusercontent.com/wealdtech/compilebug/master/signatures.go, it is 12.5MB though :) so I've derived for you a repro that generates the offending code, and then invokes
go run main.go on it as per https://play.golang.org/p/yVvr7zK5W7c or inlined below (this smaller file can then be checked into source code)

package main

import (
	"context"
	"fmt"
	"io/ioutil"
	"log"
	"os"
	"os/exec"
	"path/filepath"
	"strings"
)

var code = fmt.Sprintf(`
package main

type function struct {
	name   string
	params []string
}

var functions map[uint32]function

func main() {
     functions = make(map[uint32]function)
     %s
}`, makeFunctions(140000))

func makeFunctions(n int) string {
	sb := new(strings.Builder)
	for i := 0; i < n; i++ {
		name := fmt.Sprintf("fn-%d", i)
		fmt.Fprintf(sb, "\tfunctions[%d] = function{name: %q, params: []string{%q}}\n", i, name, name)
	}
	return sb.String()
}

func main() {
	tmpDir, err := ioutil.TempDir(os.TempDir(), "issue-33437")
	if err != nil {
		log.Fatalf("Failed to create tempdir: %v", err)
	}
	defer os.Remove(tmpDir)

	generatedMainGoFile := filepath.Join(tmpDir, "main.go")
	if err := ioutil.WriteFile(generatedMainGoFile, []byte(code), 0655); err != nil {
		log.Printf("Failed to write generated main.go file: %v", err)
		return
	}

	ctx := context.Background() // Perhaps set it to a timer.
	cmd := exec.CommandContext(ctx, "go", "run", generatedMainGoFile)
	output, err := cmd.CombinedOutput()
	if err != nil {
		log.Printf("Failed to run the command: %v\n%s", err, output)
	}
}

[odeke-em]

On my Darwin machine, it took about 50 minutes to run but it did so successfully, and here is my uname -a output

Darwin Emmanuels-MacBook-Pro-2.local 18.7.0 Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT 2019; root:xnu-4903.271.2~2/RELEASE_X86_64 x86_64

and go version

$ go version
go version devel +30da79d958 Mon Oct 7 17:19:13 2019 +0000 darwin/amd64

[Jorropo]

After a bisect this was fixed in 0202ad0 while fixing #57657.
This is available since 1.20rc3 so all currently maintained go versions thus closing as fixed even tho it were never backported.

git bisect start
# status: waiting for both good and bad commits
# bad: [6885bad7dd86880be6929c02085e5c7a67ff2887] [release-branch.go1.23] go1.23.0
git bisect bad 6885bad7dd86880be6929c02085e5c7a67ff2887
# status: waiting for good commit(s), bad commit known
# good: [2d6ee6e89a4b30c7528d2977df4e1aa86651e4e4] os: enable the close-on-exec flag for openFdAt
git bisect good 2d6ee6e89a4b30c7528d2977df4e1aa86651e4e4
# good: [3949faf72e8285622ebfdf3bd573125dcd5453d2] test: add test that was miscompiled by gccgo
git bisect good 3949faf72e8285622ebfdf3bd573125dcd5453d2
# bad: [93f10b88293f331d507cf62f4374ec809f611780] cmd/compile: fix wrong escape analysis for go/defer generic calls
git bisect bad 93f10b88293f331d507cf62f4374ec809f611780
# good: [c1a4e0fe014568501b194eb8b04309f54eee6b4c] cmd/compile: fix libfuzzer instrumentation line number
git bisect good c1a4e0fe014568501b194eb8b04309f54eee6b4c
# good: [c9a62b7e711f5c1f1a73e0c3a5b6a2e5b67033e2] os/exec: recombine goroutinePipes and userPipes
git bisect good c9a62b7e711f5c1f1a73e0c3a5b6a2e5b67033e2
# good: [dccc58e1b9d3fbc88c6a86c02f77ed0f26d07a4a] cmd/go: don't report non-go files in CompiledGoFiles
git bisect good dccc58e1b9d3fbc88c6a86c02f77ed0f26d07a4a
# bad: [78558d5e10a30c88c0b564e69963a0f0188c1dbd] archive/zip: use base offset 0 if it has a valid entry
git bisect bad 78558d5e10a30c88c0b564e69963a0f0188c1dbd
# good: [e57e673e7ff4f5b61180a07a1a73bd5bd524981a] api: promote next to go1.20
git bisect good e57e673e7ff4f5b61180a07a1a73bd5bd524981a
# bad: [0a0de0fc4241c7531860e3c1a03ab4a4643ec37f] runtime: revert use of __fork to work around Apple atfork bugs
git bisect bad 0a0de0fc4241c7531860e3c1a03ab4a4643ec37f
# good: [a5a47442502e8e42b390f458f8f91dea3de872eb] os: reenable TestReaddirSmallSeek on windows
git bisect good a5a47442502e8e42b390f458f8f91dea3de872eb
# good: [cdc73f0679c1477c262430d3476ab88c76c4a516] .github: suggest using private browsing in pkgsite template
git bisect good cdc73f0679c1477c262430d3476ab88c76c4a516
# good: [76ec91923752f69d2fb20d8b5a2e7963a116ebd6] net: fix typo in hosts.go
git bisect good 76ec91923752f69d2fb20d8b5a2e7963a116ebd6
# good: [0bbd67e52f98ce45e39e9a43e3c87a8c32aabbd2] runtime/pprof: document possibility of empty stacks
git bisect good 0bbd67e52f98ce45e39e9a43e3c87a8c32aabbd2
# good: [64519baf3802f96a813f3f35e87aefa30a5f5f73] cmd/compile/internal/pgo: add hint to missing start_line error
git bisect good 64519baf3802f96a813f3f35e87aefa30a5f5f73
# bad: [82f09b75ca181a6be0e594e1917e4d3d91934b27] os/exec: avoid leaking an exec.Cmd in TestWaitInterrupt
git bisect bad 82f09b75ca181a6be0e594e1917e4d3d91934b27