Cherry-pick #9539 to 6.x: Fix race detector detected race in heartbeat #9559
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The race detector found this: ``` 15:40:04 make unit-tests 15:40:04 make[1]: Entering directory '/var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat' 15:40:05 mkdir -p /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/build/coverage 15:40:05 # gotestcover is needed to fetch coverage for multiple packages 15:40:05 go get github.com/elastic/beats/vendor/github.com/pierrre/gotestcover 15:40:06 # testify is needed for unit and integration tests 15:40:06 go get github.com/elastic/beats/vendor/github.com/stretchr/testify/assert 15:40:07 go test -i github.com/elastic/beats/heartbeat github.com/elastic/beats/heartbeat/beater github.com/elastic/beats/heartbeat/cmd github.com/elastic/beats/heartbeat/config github.com/elastic/beats/heartbeat/hbtest github.com/elastic/beats/heartbeat/include github.com/elastic/beats/heartbeat/look github.com/elastic/beats/heartbeat/monitors github.com/elastic/beats/heartbeat/monitors/active/dialchain github.com/elastic/beats/heartbeat/monitors/active/http github.com/elastic/beats/heartbeat/monitors/active/icmp github.com/elastic/beats/heartbeat/monitors/active/tcp github.com/elastic/beats/heartbeat/monitors/defaults github.com/elastic/beats/heartbeat/reason github.com/elastic/beats/heartbeat/scheduler github.com/elastic/beats/heartbeat/scheduler/schedule github.com/elastic/beats/heartbeat/scheduler/schedule/cron github.com/elastic/beats/heartbeat/scripts/generate_monitor github.com/elastic/beats/heartbeat/watcher 15:40:08 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/bin/gotestcover -race -coverprofile=/var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/build/coverage/unit.cov github.com/elastic/beats/heartbeat github.com/elastic/beats/heartbeat/beater github.com/elastic/beats/heartbeat/cmd github.com/elastic/beats/heartbeat/config github.com/elastic/beats/heartbeat/hbtest github.com/elastic/beats/heartbeat/include github.com/elastic/beats/heartbeat/look github.com/elastic/beats/heartbeat/monitors github.com/elastic/beats/heartbeat/monitors/active/dialchain github.com/elastic/beats/heartbeat/monitors/active/http github.com/elastic/beats/heartbeat/monitors/active/icmp github.com/elastic/beats/heartbeat/monitors/active/tcp github.com/elastic/beats/heartbeat/monitors/defaults github.com/elastic/beats/heartbeat/reason github.com/elastic/beats/heartbeat/scheduler github.com/elastic/beats/heartbeat/scheduler/schedule github.com/elastic/beats/heartbeat/scheduler/schedule/cron github.com/elastic/beats/heartbeat/scripts/generate_monitor github.com/elastic/beats/heartbeat/watcher 15:40:43 ? github.com/elastic/beats/heartbeat/beater [no test files] 15:40:43 ok github.com/elastic/beats/heartbeat/config 1.136s coverage: 0.0% of statements [no tests to run] 15:40:45 ? github.com/elastic/beats/heartbeat/include [no test files] 15:40:50 ? github.com/elastic/beats/heartbeat/hbtest [no test files] 15:40:56 ok github.com/elastic/beats/heartbeat/look 1.107s coverage: 100.0% of statements 15:40:59 ? github.com/elastic/beats/heartbeat/monitors/active/dialchain [no test files] 15:41:04 ok github.com/elastic/beats/heartbeat/monitors 1.113s coverage: 33.1% of statements 15:41:06 ? github.com/elastic/beats/heartbeat/monitors/active/icmp [no test files] 15:41:15 command [go test -race -cover -coverprofile /tmp/gotestcover-598500574 github.com/elastic/beats/heartbeat/monitors/active/http]: exit status 1 15:41:15 ================== 15:41:15 WARNING: DATA RACE 15:41:15 Read at 0x00c0002f8e00 by goroutine 47: 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.createPingFactory.func1() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/task.go:185 +0x91b 15:41:15 github.com/elastic/beats/heartbeat/monitors.MakePingIPFactory.func1.1() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:194 +0x49 15:41:15 github.com/elastic/beats/heartbeat/monitors.MakeSimpleCont.func1() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:184 +0x41 15:41:15 github.com/elastic/beats/heartbeat/monitors.funcTask.Run() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:440 +0x38 15:41:15 github.com/elastic/beats/heartbeat/monitors.WithFields.func1() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:398 +0x6a 15:41:15 github.com/elastic/beats/heartbeat/monitors.funcTask.Run() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:440 +0x38 15:41:15 github.com/elastic/beats/heartbeat/monitors.TaskRunner.Run-fm() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:168 +0x4a 15:41:15 github.com/elastic/beats/heartbeat/monitors.annotated.func1() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:140 +0xdc 15:41:15 github.com/elastic/beats/heartbeat/monitors.MakeJob.func1() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:127 +0x15b 15:41:15 github.com/elastic/beats/heartbeat/monitors.(*funcJob).Run() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:438 +0xb2 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.testTLSRequest() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/http_test.go:69 +0x46b 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.runHTTPSServerCheck() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/http_test.go:278 +0x526 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.TestHTTPSx509Auth() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/http_test.go:326 +0x70d 15:41:15 testing.tRunner() 15:41:15 /var/lib/jenkins/.gvm/versions/go1.11.2.linux.amd64/src/testing/testing.go:827 +0x162 15:41:15 15:41:15 Previous write at 0x00c0002f8e00 by goroutine 38: 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.createPingFactory.func1.1() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/task.go:166 +0x92 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.call() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/simple_transp.go:221 +0x7d 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.(*SimpleTransport).sigStartWrite() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/simple_transp.go:216 +0x68 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.(*SimpleTransport).writeRequest() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/simple_transp.go:155 +0x1ca 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.(*SimpleTransport).RoundTrip.func1() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/simple_transp.go:106 +0x72 15:41:15 15:41:15 Goroutine 47 (running) created at: 15:41:15 testing.(*T).Run() 15:41:15 /var/lib/jenkins/.gvm/versions/go1.11.2.linux.amd64/src/testing/testing.go:878 +0x650 15:41:15 testing.runTests.func1() 15:41:15 /var/lib/jenkins/.gvm/versions/go1.11.2.linux.amd64/src/testing/testing.go:1119 +0xa8 15:41:15 testing.tRunner() 15:41:15 /var/lib/jenkins/.gvm/versions/go1.11.2.linux.amd64/src/testing/testing.go:827 +0x162 15:41:15 testing.runTests() 15:41:15 /var/lib/jenkins/.gvm/versions/go1.11.2.linux.amd64/src/testing/testing.go:1117 +0x4ee 15:41:15 testing.(*M).Run() 15:41:15 /var/lib/jenkins/.gvm/versions/go1.11.2.linux.amd64/src/testing/testing.go:1034 +0x2ee 15:41:15 main.main() 15:41:15 _testmain.go:122 +0x332 15:41:15 15:41:15 Goroutine 38 (running) created at: 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.(*SimpleTransport).RoundTrip() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/simple_transp.go:105 +0x31f 15:41:15 net/http.send() 15:41:15 /var/lib/jenkins/.gvm/versions/go1.11.2.linux.amd64/src/net/http/client.go:250 +0x304 15:41:15 net/http.(*Client).send() 15:41:15 /var/lib/jenkins/.gvm/versions/go1.11.2.linux.amd64/src/net/http/client.go:174 +0x1ca 15:41:15 net/http.(*Client).do() 15:41:15 /var/lib/jenkins/.gvm/versions/go1.11.2.linux.amd64/src/net/http/client.go:641 +0x531 15:41:15 net/http.(*Client).Do() 15:41:15 /var/lib/jenkins/.gvm/versions/go1.11.2.linux.amd64/src/net/http/client.go:509 +0x42 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.execRequest() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/task.go:253 +0xdf 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.execPing() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/task.go:227 +0x160 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.createPingFactory.func1() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/task.go:172 +0x7b8 15:41:15 github.com/elastic/beats/heartbeat/monitors.MakePingIPFactory.func1.1() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:194 +0x49 15:41:15 github.com/elastic/beats/heartbeat/monitors.MakeSimpleCont.func1() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:184 +0x41 15:41:15 github.com/elastic/beats/heartbeat/monitors.funcTask.Run() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:440 +0x38 15:41:15 github.com/elastic/beats/heartbeat/monitors.WithFields.func1() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:398 +0x6a 15:41:15 github.com/elastic/beats/heartbeat/monitors.funcTask.Run() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:440 +0x38 15:41:15 github.com/elastic/beats/heartbeat/monitors.TaskRunner.Run-fm() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:168 +0x4a 15:41:15 github.com/elastic/beats/heartbeat/monitors.annotated.func1() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:140 +0xdc 15:41:15 github.com/elastic/beats/heartbeat/monitors.MakeJob.func1() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:127 +0x15b 15:41:15 github.com/elastic/beats/heartbeat/monitors.(*funcJob).Run() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/util.go:438 +0xb2 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.testTLSRequest() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/http_test.go:69 +0x46b 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.runHTTPSServerCheck() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/http_test.go:278 +0x526 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.TestHTTPSx509Auth() 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/http_test.go:326 +0x70d 15:41:15 testing.tRunner() 15:41:15 /var/lib/jenkins/.gvm/versions/go1.11.2.linux.amd64/src/testing/testing.go:827 +0x162 15:41:15 ================== 15:41:15 panic: runtime error: invalid memory address or nil pointer dereference 15:41:15 [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xba7895] 15:41:15 15:41:15 goroutine 906 [running]: 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.(*SimpleTransport).readResponse(0xc000142c00, 0xd9bda0, 0xc000234700, 0xc000172300, 0xc000471f00, 0xd9bec0, 0xc0001ce120, 0xc000387c00) 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/simple_transp.go:187 +0x235 15:41:15 github.com/elastic/beats/heartbeat/monitors/active/http.(*SimpleTransport).RoundTrip.func2(0xc000142c00, 0xd9bda0, 0xc000234700, 0xc000172300, 0xc000172300, 0xc0004340c0) 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/simple_transp.go:111 +0x82 15:41:15 created by github.com/elastic/beats/heartbeat/monitors/active/http.(*SimpleTransport).RoundTrip 15:41:15 /var/lib/jenkins/workspace/elastic+beats+pull-request+multijob-linux/beat/heartbeat/label/ubuntu/src/github.com/elastic/beats/heartbeat/monitors/active/http/simple_transp.go:110 +0x397 15:41:15 FAIL github.com/elastic/beats/heartbeat/monitors/active/http 3.677s 15:41:18 ? github.com/elastic/beats/heartbeat/monitors/defaults [no test files] 15:41:20 ? github.com/elastic/beats/heartbeat/reason [no test files] 15:41:22 ok github.com/elastic/beats/heartbeat/monitors/active/tcp 2.149s coverage: 45.6% of statements 15:41:28 ok github.com/elastic/beats/heartbeat/scheduler 1.192s coverage: 76.4% of statements 15:41:29 ? github.com/elastic/beats/heartbeat/scheduler/schedule/cron [no test files] 15:41:31 ? github.com/elastic/beats/heartbeat/scripts/generate_monitor [no test files] 15:41:31 ok github.com/elastic/beats/heartbeat/scheduler/schedule 1.156s coverage: 93.8% of statements 15:41:33 ? github.com/elastic/beats/heartbeat/watcher [no test files] 15:41:41 ? github.com/elastic/beats/heartbeat/cmd [no test files] 15:41:45 ok github.com/elastic/beats/heartbeat 1.054s coverage: 0.0% of statements 15:41:45 test failed 15:41:45 make[1]: *** [unit-tests] Error 1 15:41:45 ../libbeat/scripts/Makefile:181: recipe for target 'unit-tests' failed 15:41:45 make: *** [testsuite] Error 2 ``` (cherry picked from commit c3d9282)
ruflin
approved these changes
Dec 17, 2018
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherry-pick of PR #9539 to 6.x branch. Original message:
The race detector found this race. It's blocking #9408