bazel: use bash with fallback $PATH consisting of basic shell utils

[malt3]

Description of changes

Fixes #265040. Ensure any shell action has a minimal set of tools available (even for remote builds).

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.11 Release Notes (or backporting 23.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[uri-canva]

Sounds ok to me.

[olebedev]

LGTM 👍

[mweinelt]

Bisect thinks this change broke python311.pkgs.jaxlib.

3576806d7edc615b2e89ee4f845d67c5b6c7ecc1 is the first bad commit
commit 3576806d7edc615b2e89ee4f845d67c5b6c7ecc1
Author: Malte Poll <[email protected]>
Date:   Thu Nov 2 13:13:00 2023 +0100

    bazel: use bash with fallback $PATH consisting of basic shell utils

 .../tools/build-managers/bazel/bazel_6/default.nix  | 21 ++++++++++++++++-----
 1 file changed, 16 insertions(+), 5 deletions(-)
bisect found first bad commit

bazel-build-jaxlib> building '/nix/store/dap85k2hizfk5wyg9qxcr95w06ig3307-bazel-build-jaxlib-0.4.19.drv'
bazel-build-jaxlib> unpacking sources
bazel-build-jaxlib> unpacking source archive /nix/store/j4bkh2nlm62pjxp7is00g2bcp608xb2b-source
bazel-build-jaxlib> source root is source
bazel-build-jaxlib> patching sources
bazel-build-jaxlib> updateAutotoolsGnuConfigScriptsPhase
bazel-build-jaxlib> configuring
bazel-build-jaxlib> no configure script, doing nothing
bazel-build-jaxlib> building
bazel-build-jaxlib> Extracting Bazel installation...
bazel-build-jaxlib> INFO: Options provided by the client:
bazel-build-jaxlib>   Inherited 'common' options: --isatty=1 --terminal_columns=0
bazel-build-jaxlib> INFO: Reading rc options for 'run' from /build/source/.bazelrc:
bazel-build-jaxlib>   Inherited 'common' options: --experimental_repo_remote_exec
bazel-build-jaxlib> INFO: Reading rc options for 'run' from /nix/store/v9x5zmj6ai0c5vjiv4yfjb0nqafiiga2-bazel-rc:
bazel-build-jaxlib>   Inherited 'build' options: --distdir=/nix/store/m8y1s64j25am1b60ihzk11wprr2wmjra-bazel-deps --extra_toolchains=@bazel_tools//tools/jdk:nonprebuilt_toolchain_definition --tool_java_runtime_version=local_jdk_11 --java_runtime_version=local_jdk_11
bazel-build-jaxlib> INFO: Reading rc options for 'run' from /build/source/.bazelrc:
bazel-build-jaxlib>   Inherited 'build' options: --nocheck_visibility --apple_platform_type=macos --macos_minimum_os=10.14 --announce_rc --define open_source_build=true --spawn_strategy=standalone --enable_platform_specific_config --experimental_cc_shared_library --define=no_aws_support=true --define=no_gcp_support=true --define=no_hdfs_support=true --define=no_kafka_support=true --define=no_ignite_support=true --define=grpc_no_ares=true --define=tsl_link_protobuf=true -c opt --config=short_logs --copt=-DMLIR_PYTHON_PACKAGE_PREFIX=jaxlib.mlir. --@xla//xla/python:enable_gpu=false
bazel-build-jaxlib> INFO: Reading rc options for 'run' from /build/source/.jax_configure.bazelrc:
bazel-build-jaxlib>   Inherited 'build' options: --strategy=Genrule=standalone --repo_env PYTHON_BIN_PATH=/nix/store/ffll6glz3gwx342z0ch8wx30p5cnqz1z-python3-3.11.5/bin/python --action_env=PYENV_ROOT --python_path=/nix/store/ffll6glz3gwx342z0ch8wx30p5cnqz1z-python3-3.11.5/bin/python --distinct_host_configuration=false --define PROTOBUF_INCLUDE_PATH=/nix/store/5prkzfpsimriksa691j4ip7w90gn4ca7-protobuf-3.24.3/include --config=mkl_open_source_only
bazel-build-jaxlib> INFO: Found applicable config definition build:short_logs in file /build/source/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
bazel-build-jaxlib> INFO: Found applicable config definition build:mkl_open_source_only in file /build/source/.bazelrc: --define=tensorflow_mkldnn_contraction_kernel=1
bazel-build-jaxlib> INFO: Found applicable config definition build:linux in file /build/source/.bazelrc: --config=posix --copt=-Wno-unknown-warning-option --copt=-Wno-stringop-truncation --copt=-Wno-array-parameter
bazel-build-jaxlib> INFO: Found applicable config definition build:posix in file /build/source/.bazelrc: --copt=-fvisibility=hidden --copt=-Wno-sign-compare --cxxopt=-std=c++17 --host_cxxopt=-std=c++17
bazel-build-jaxlib> Loading: 
bazel-build-jaxlib> DEBUG: /build/output/external/xla/third_party/repo.bzl:132:14: 
bazel-build-jaxlib> Warning: skipping import of repository 'llvm-raw' because it already exists.
bazel-build-jaxlib> Loading: 
bazel-build-jaxlib> Loading: 1 packages loaded
bazel-build-jaxlib> Analyzing: target //jaxlib/tools:build_wheel (2 packages loaded, 0 targets configured)
bazel-build-jaxlib> Analyzing: target //jaxlib/tools:build_wheel (65 packages loaded, 392 targets configured)
bazel-build-jaxlib> Analyzing: target //jaxlib/tools:build_wheel (189 packages loaded, 8992 targets configured)
bazel-build-jaxlib> INFO: Analyzed target //jaxlib/tools:build_wheel (200 packages loaded, 13198 targets configured).
bazel-build-jaxlib> INFO: Found 1 target...
bazel-build-jaxlib> [0 / 20] [Prepa] BazelWorkspaceStatusAction stable-status.txt ... (2 actions, 1 running)
bazel-build-jaxlib> ERROR: /build/output/external/local_config_python/BUILD:78:8: Executing genrule @local_config_python//:python_include failed: (Exit 1): bash failed: error executing command (from target @local_config_python//:python_include) /nix/store/j2l90z41r5c85zpfbj91dsxycm6mm8s9-bash/bin/bash -c ... (remaining 1 argument skipped)
bazel-build-jaxlib> src/main/tools/process-wrapper-legacy.cc:80: "execvp(/nix/store/j2l90z41r5c85zpfbj91dsxycm6mm8s9-bash/bin/bash, ...)": No such file or directory
bazel-build-jaxlib> ERROR: /build/output/external/xla/third_party/py/numpy/BUILD:16:8: Executing genrule @xla//third_party/py/numpy:dummy failed: (Exit 1): bash failed: error executing command (from target @xla//third_party/py/numpy:dummy) /nix/store/j2l90z41r5c85zpfbj91dsxycm6mm8s9-bash/bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; touch bazel-out/k8-opt/bin/external/xla/third_party/py/numpy/tf_numpy_dummy.py'
bazel-build-jaxlib> src/main/tools/process-wrapper-legacy.cc:80: "execvp(/nix/store/j2l90z41r5c85zpfbj91dsxycm6mm8s9-bash/bin/bash, ...)": No such file or directory
bazel-build-jaxlib> Target //jaxlib/tools:build_wheel failed to build
bazel-build-jaxlib> Use --verbose_failures to see the command lines of failed build steps.
bazel-build-jaxlib> INFO: Elapsed time: 4.378s, Critical Path: 0.03s
bazel-build-jaxlib> INFO: 57 processes: 57 internal.
bazel-build-jaxlib> FAILED: Build did NOT complete successfully
bazel-build-jaxlib> ERROR: Build failed. Not running target
bazel-build-jaxlib> 
error: builder for '/nix/store/dap85k2hizfk5wyg9qxcr95w06ig3307-bazel-build-jaxlib-0.4.19.drv' failed with exit code 1;

[bjornfor]

Revert PR here: #266842

[malt3]

bazel-build-jaxlib> src/main/tools/process-wrapper-legacy.cc:80: "execvp(/nix/store/j2l90z41r5c85zpfbj91dsxycm6mm8s9-bash/bin/bash, ...)": No such file or directory

I'm pretty sure this is my bashWithDefaultShellUtils. I forgot properly add it as a buildInput of Bazel so it wasn't guaranteed to be available at runtime when bazel is used to build other derivations.