cockroach 19.1.1

[ajkr]

Update cockroachdb formula to v19.1.1

[chenrui333]

relates to #40048
Also relates to cockroachdb/cockroach#35637

[benesch]

This built properly, but the formula's smoke test failed. Did the behavior of cockroach start --background change in 19.1? The test does something like this:

cockroach start --background
cockroach sql -e 'SELECT 1';

It's the cockroach sql command that's failing because the server's not yet running. Does --background no longer wait for the server to start listening before daemonizing?

[benesch]

Actually, I suppose it's more likely that something's gone wrong during server bootup. Unfortunately any such error messages are buried inside start.out, which isn't shown in the error log. Someone will have to adjust the test to print out those errors.

[zbeekman]

@ajkr if that doesn't fix the test, you may try adding a few seconds of sleep before trying to contact the server process.

[zbeekman]

Test still failing. Try following advice in output, and, maybe adding a few seconds of sleep before attempting to spawn the process that connects to the server...

darwin18:/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/universal-darwin18:/usr/local/Homebrew/Library/Homebrew -- /usr/local/Homebrew/Library/Homebrew/test.rb /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/cockroach.rb --verbose
==> /usr/local/Cellar/cockroach/19.1.1/bin/cockroach start --insecure --background --advertise-host=localhost &> start.out
==> /usr/local/Cellar/cockroach/19.1.1/bin/cockroach quit --insecure
Error: cannot dial server.
Is the server running?
If the server is running, check --host client-side and --advertise server-side.

Failed to connect to the node: initial connection heartbeat failed: rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: Error while dialing dial tcp [::1]:26257: connect: connection refused"
Failed running "quit"
Error: cockroach: failed

[bdarnell]

Sleeping shouldn't be necessary (the test uses --background instead of & because the former waits to open all its ports etc before going into the background). We need to see what's in start.out (so I guess change the &> to a tee).

When I try to reproduce this build locally (on two different machines), I get a lot of weird c++ errors that look like they're problems in how the c++ standard library is getting compiled:

In file included from /tmp/cockroach-20190520-38469-hvi3ra/cockroach-v19.1.1/src/github.com/cockroachdb/cockroach/c-deps/protobuf/src/google/protobuf/stubs/hash.h:146:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/unordered_map:384:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/__hash_table:19:
/Library/Developer/CommandLineTools/usr/include/c++/v1/cmath:316:9: error: no member named 'isinf' in the global namespace
using ::isinf;
      ~~^
/Library/Developer/CommandLineTools/usr/include/c++/v1/cmath:317:9: error: no member named 'isnan' in the global namespace
using ::isnan;
      ~~^

[zbeekman]

When I try to reproduce this build locally (on two different machines), I get a lot of weird c++ errors that look like they're problems in how the c++ standard library is getting compiled:

Which xcode do you have? You may need to install the C++ headers:

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

[bdarnell]

That worked. Why was this manual step necessary? Are we missing a dependency? (one of the two machines I tested on had a brand new installation of brew; it had previously only used macports)

I have the latest xcode and CLI tools (or at least that's what software update and xcode-select --install tell me). I'm able to build c++ outside of a brew context.

And FWIW, brew test ./Formula/cockroach.rb works on my machine, so we'll need to get that output to see why it's failing CI.

[zbeekman]

That worked. Why was this manual step necessary? Are we missing a dependency?

No, it's because Apple messes with the C++ libs and tooling all the time. IIRC, that installs headers that Apple considers deprecated or something like that. ¯\_(ツ)_/¯

Edit: IIRC, it's also specific to Mojave, and happens everytime you upgrade xcode.

[bdarnell]

Installing the package worked so I don't want to dwell on it here, but one last observation. This SO question suggests that brew tries to detect this condition and give a prompt for it; this didn't happen in my case. I still have a second machine on which that package has not been installed if it would be a useful test case for making that warning more reliable.

[benesch]

I'm hopeful that #40132 will help us get to the bottom of this.

[benesch]

Not sure what to make of this:

* ERROR: cockroach server exited with error: failed to create engines: invalid argument

[bdarnell]

An inscrutable error at this stage generally means we're either running on a filesystem with incomplete posix support (such as whatever WSL uses), or in this case, maybe sandbox-exec is blocking something.

It looks like one way we could produce "failed to create engines: invalid argument" without additional context is if the setrlimit system call fails. Most other error paths seem to have some more context added to them.

We have comments indicating that macos would sometimes fail to return an error if you try to set the file descriptor limit to a higher value than allowed. Maybe now it's behaving correctly, but we break if that error is ever hit. Is there any way to increase the allowed file descriptor limit in brew CI (the tests pass when run locally)? If not, we'll probably have to wait until 19.1.2 to get an updated package into brew.

[bdarnell]

See also golang/go#30401

[benesch]

@bdarnell hit the nail on the head. I've gotten CI to pass in #40132 with a patch to the rlimit code.

[ajkr]

Thanks for solving it, Nikhil and Ben. Will close this one in favor of #40132.