Bazel 0.13.0 adds a strict requirement to have a c compiler locally (even for RE)

[nlopezgi]

Description of the problem / feature request:

Bazel 0.13.0 adds a strict requirement to have a c compiler installed, even for remote builds (which should not require a c compiler locally installed - and did not with bazel 0.12.0 or earlier)

Feature requests: what underlying problem are you trying to solve with this feature?

started noticing issues with their remote execution builds since Bazel 0.13.0 was released.
If no c compiler is present locally, the following error is produced:

ERROR: Analysis of target 'xxx' failed; build aborted: no such package '@local_config_cc//': Traceback (most recent call last):
       File ".../external/bazel_tools/tools/cpp/cc_configure.bzl", line 41
               configure_unix_toolchain(repository_ctx, cpu_value, overriden...)
       File ".../external/bazel_tools/tools/cpp/unix_cc_configure.bzl", line 413, in configure_unix_toolchain
               find_cc(repository_ctx, overriden_tools)
       File ".../external/bazel_tools/tools/cpp/unix_cc_configure.bzl", line 404, in find_cc
               fail(("Cannot find gcc or CC%s, eithe...))
Cannot find gcc or CC, either correct your path or set the CC environment variable

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Run bazel from a container that has no C compiler installed, using appropriate remote execution flags.
Example repro steps (with RE instance specific details removed):

1. Prepare the debian9 container

gcloud docker -- pull gcr.io/cloud-marketplace/google/debian9@sha256:6b3aa04751aa2ac3b0c7be4ee71148b66d693ad212ce6d3244bd2a2a147f314a

2. Run the container

docker run -v /temp:/temp -it gcr.io/cloud-marketplace/google/debian9@sha256:6b3aa04751aa2ac3b0c7be4ee71148b66d693ad212ce6d3244bd2a2a147f314a /bin/bash

3. Install unzip, wget, git (inside the container)

apt-get update && apt-get install -y --no-install-recommends unzip wget git

4. Install bazel 0.13.0 (w/o using apt-get)

wget --no-verbose https://github.com/bazelbuild/bazel/releases/download/0.13.0/bazel-0.13.0-installer-linux-x86_64.sh
chmod 777 bazel-0.13.0-installer-linux-x86_64.sh
./bazel-0.13.0-installer-linux-x86_64.sh

5. Pull a project from github (e.g., Abseil-cpp)

git clone https://github.com/abseil/abseil-cpp.git
cd abseil-cpp/

6. Try to build it with Remote execution

bazel --host_jvm_args=-Dbazel.DigestFunction=SHA256 --nomaster_bazelrc test --verbose_failures --jobs=50 --spawn_strategy=remote --strategy=Javac=remote --strategy=Closure=remote --genrule_strategy=remote --experimental_strict_action_env=true --host_javabase=@bazel_toolchains//configs/debian8_clang/0.3.0:jdk8 --javabase=@bazel_toolchains//configs/debian8_clang/0.3.0:jdk8 --tls_enabled --auth_enabled --auth_scope= --auth_credentials= --remote_executor= --remote_cache= --remote_local_fallback=False --remote_accept_cached=True --experimental_remote_platform_override='properties:{name:"container-image" value:"docker://gcr.io/cloud-marketplace/google/rbe-debian8@sha256:d84a7de5175a22505209f56b02f1da20ccec64880da09ee38eaef3670fbd2a56" }' --remote_timeout=3600 --remote_instance_name= --crosstool_top=@bazel_toolchains//configs/debian8_clang/0.3.0/bazel_0.10.0:toolchain --bes_timeout=60s --bes_backend= --project_id= --define EXECUTOR=remote -- //absl/...

7. Build should fail with error indicated above

What operating system are you running Bazel on?

debian9 (http://gcr.io/cloud-marketplace/google/debian9)

What's the output of bazel info release?

release 0.13.0

Have you found anything relevant by searching the web?

consulted with katre@ about the issue. From conversation:
Before 0.13.0, not having CC set caused an error only if you actually used CC compilation, which [does not happen with remote execution]. This is because Bazel never tries to load the @local_config_cc// package [with RE].
After 0.13.0, the toolchains from cc autoconfiguration are always registered, so Bazel always tries to load @local_config_cc//.

[katre]

All of the other arguments are not needed, this error is reproducible by running only:

bazel build //absl/...

[katre]

Okay, managed to build bazel from head and test that, it's a similar error:

ERROR: in target '//external:cc_toolchain': no such package '@local_config_cc//': Traceback (most recent call last):
	File "/root/.cache/bazel/_bazel_root/7eca63b212bb77491fc771313ed1b4be/external/bazel_tools/tools/cpp/cc_configure.bzl", line 41
		configure_unix_toolchain(repository_ctx, cpu_value, overriden...)
	File "/root/.cache/bazel/_bazel_root/7eca63b212bb77491fc771313ed1b4be/external/bazel_tools/tools/cpp/unix_cc_configure.bzl", line 429, in configure_unix_toolchain
		_find_generic(repository_ctx, "gcc", "CC", overriden...)
	File "/root/.cache/bazel/_bazel_root/7eca63b212bb77491fc771313ed1b4be/external/bazel_tools/tools/cpp/unix_cc_configure.bzl", line 418, in _find_generic
		auto_configure_fail(msg)
	File "/root/.cache/bazel/_bazel_root/7eca63b212bb77491fc771313ed1b4be/external/bazel_tools/tools/cpp/lib_cc_configure.bzl", line 84, in auto_configure_fail
		fail(("\n%sAuto-Configuration Error:%...)))

[katre]

Okay, debugged further. Once I remove the following lines from tools/cpp/cc_configure.bzl, the build continues until a cc_library target is built:

 native.bind(name="cc_toolchain", actual="@local_config_cc//:toolchain")
  native.register_toolchains(
      # Use register_toolchain's target pattern expansion to register all toolchains in the package.
      "@local_config_cc//:all")

One of these I added in f5c8c0b, the first is there since the file was first created.

[katre]

@mhlopko, do we intend to support the case of Bazel with no CC compiler? If so, we need a better way to entirely turn off cc_configure, I think.

[katre]

This failure also exists in 0.12.0. I'm not sure we've ever supported this case very well.

[katre]

I see this does work with 0.12.0, I had misunderstood the specific problem. In 0.12.0, you cannot build cc_library targets, but everything else works fine even if cc autoconfiguration fails.

[benjaminp]

#4691 (comment) gave me the impression that not requiring a C++ toolchain if no C++ targets are requested is a goal.

[nlopezgi]

please note that for remote execution (or for local docker sandbox), even if c++ targets are requested, having a c/c++ compiler installed locally is not a requirement, so having any kind of hard check that there is a local c compiler (unless the local cc toolchain is explicitly requested) is problematic

[hlopko]

Would having a environment variable (e.g. DISABLE_CC_AUTOCONF) that will disable cc_configre solve this issue?

[nlopezgi]

although its not the most elegant solution, I think it should work and sounds easy enough to implement.

[hlopko]

@katre, will the env var thing work when the WORKSPACE file will still have the register_toolchains entry? What we want is a way to disable this statement: https://source.bazel.build/bazel/+/master:tools/cpp/cc_configure.bzl;l=84?q=register_toolchains correct?

[nlopezgi]

friendly ping @katre

[katre]

Sorry, was out on vacation until today.

@mhlopko, as long as the @local_config_cc repository exists, even if there are no actual toolchains in it, the register_toolchains call is fine. It takes a target pattern, and if the pattern evaluates to an empty list, that's fine. The problem right now is that the target pattern parsing fails because @local_config_cc doesn't exists due to the missing CC setup.

As long as the environment variable causes an empty @local_config_cc to be generated, this should be fine.

[hlopko]

@nlopezgi I leave the decision whether to cherrypick to 0.14 to you and to @lfpino, current release manager.

[lfpino]

Hi all, please post the cherrypick requests in #5056 with a summary of the regression, I haven't really followed the whole discussion. Thanks!

[nlopezgi]

Commented on #5056 with a short summary. I think @mhlopko 's fix is almost in. Please let me know if this can be cherry-picked. Thanks