test/ci: support for clang asan/tsan in build and travis. (#904)

* Add --config={clang-asan,clang-tsan} configuration option to Bazel CLI for when building with
  clang-5.0.

* Add clang-asan, which also checks some static initialization order stuff.

* Add ubsan to asan/clang-asan configs, the two play nicely together.

* Switch bazel.asan from gcc-4.9+asan to clang-5.0+clang-asan.

* Add bazel.tsan CI target, enabled in .travis.yml.

MSAN is not included due to the issue discussed at #443.

.travis.yml

@@ -9,6 +9,7 @@ matrix:
 env:
   - TEST_TYPE=bazel.release
   - TEST_TYPE=bazel.asan
+  - TEST_TYPE=bazel.tsan
   - TEST_TYPE=bazel.coverage
   - TEST_TYPE=check_format
   - TEST_TYPE=docs

bazel/README.md

@@ -147,13 +147,32 @@ You can use the `-c <compilation_mode>` flag to control this, e.g.
 ```
 bazel build -c opt //source/exe:envoy-static
 ```
-To build and run tests with the compiler's address sanitizer (ASAN) enabled:
+
+
+## Sanitizers
+
+To build and run tests with the gcc-4.9 compiler's [address sanitizer
+(ASAN)](https://github.com/google/sanitizers/wiki/AddressSanitizer) and
+[undefined behavior
+(UBSAN)](https://developers.redhat.com/blog/2014/10/16/gcc-undefined-behavior-sanitizer-ubsan) sanitizer enabled:
 
 ```
 bazel test -c dbg --config=asan //test/...
 ```
 
-The ASAN failure stack traces include line numbers as a results of running ASAN with a `dbg` build above.
+The ASAN failure stack traces include line numbers as a result of running ASAN with a `dbg` build above.
+
+If you have clang-5.0, additional checks are provided with:
+
+```
+bazel test -c dbg --config=clang-asan //test/...
+```
+
+Similarly, for [thread sanitizer (TSAN)](https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual) testing:
+
+```
+bazel test -c dbg --config=clang-tsan //test/...
+```
 
 # Release builds
 

ci/README.md

@@ -46,10 +46,11 @@ The artifacts can be found in `/tmp/envoy-docker-build/envoy/source/exe/envoy` (
 
 The `./ci/run_envoy_docker.sh './ci/do_ci.sh <TARGET>'` targets are:
 
-* `bazel.asan` &mdash; build and run tests under `-c dbg --config=asan`.
-* `bazel.dev` &mdash; build Envoy static binary and run tests under `-c fastbuild`.
-* `bazel.release` &mdash; build Envoy static binary and run tests under `-c opt`.
-* `bazel.release.server_only` &mdash; build Envoy static binary under `-c opt`.
-* `bazel.coverage` &mdash; build and run tests under `-c dbg`, generating coverage information in `<SOURCE_DIR>/generated/coverage/coverage.html`.
+* `bazel.asan` &mdash; build and run tests under `-c dbg --config=clang-asan` with clang-5.0.
+* `bazel.dev` &mdash; build Envoy static binary and run tests under `-c fastbuild` with gcc-4.9.
+* `bazel.release` &mdash; build Envoy static binary and run tests under `-c opt` with gcc-4.9.
+* `bazel.release.server_only` &mdash; build Envoy static binary under `-c opt` with gcc-4.9.
+* `bazel.coverage` &mdash; build and run tests under `-c dbg` with gcc-4.9, generating coverage information in `<SOURCE_DIR>/generated/coverage/coverage.html`.
+* `bazel.tsan` &mdash; build and run tests under `-c dbg --config=clang-tsan` with clang-5.0.
 * `check_format`&mdash; run `clang-format` 3.6 and `buildifier` on entire source tree.
 * `fix_format`&mdash; run and enforce `clang-format` 3.6 and `buildifier` on entire source tree.

ci/build_setup.sh

@@ -4,15 +4,28 @@
 
 set -e
 
-export CC=gcc-4.9
-export CXX=g++-4.9
 export HEAPCHECK=normal
 export PPROF_PATH=/thirdparty_build/bin/pprof
 
 NUM_CPUS=`grep -c ^processor /proc/cpuinfo`
 
 export ENVOY_SRCDIR=/source
 
+function setup_gcc_toolchain() {
+  export CC=gcc-4.9
+  export CXX=g++-4.9
+  echo "$CC/$CXX toolchain configured"
+}
+
+function setup_clang_toolchain() {
+  export CC=clang-5.0
+  export CXX=clang++-5.0
+  # Ensure clang doesn't use a C++ ABI incompatible with the prebuilts.
+  export BAZEL_BUILD_OPTIONS="${BAZEL_BUILD_OPTIONS} --copt=-D_GLIBCXX_USE_CXX11_ABI=0"
+  export BAZEL_TEST_OPTIONS="${BAZEL_TEST_OPTIONS} --copt=-D_GLIBCXX_USE_CXX11_ABI=0"
+  echo "$CC/$CXX toolchain configured"
+}
+
 # Create a fake home. Python site libs tries to do getpwuid(3) if we don't and the CI
 # Docker image gets confused as it has no passwd entry when running non-root
 # unless we do this.

ci/do_ci.sh

@@ -19,24 +19,35 @@ function bazel_release_binary_build() {
 }
 
 if [[ "$1" == "bazel.release" ]]; then
+  setup_gcc_toolchain
   echo "bazel release build with tests..."
   bazel_release_binary_build
   echo "Testing..."
   bazel --batch test ${BAZEL_TEST_OPTIONS} -c opt //test/...
   exit 0
 elif [[ "$1" == "bazel.release.server_only" ]]; then
+  setup_gcc_toolchain
   echo "bazel release build..."
   bazel_release_binary_build
   exit 0
 elif [[ "$1" == "bazel.asan" ]]; then
-  echo "bazel ASAN debug build with tests..."
+  setup_clang_toolchain
+  echo "bazel ASAN/UBSAN debug build with tests..."
   cd "${ENVOY_FILTER_EXAMPLE_SRCDIR}"
   echo "Building and testing..."
-  # TODO(htuch): This should switch to using clang when available.
-  bazel --batch test ${BAZEL_TEST_OPTIONS} -c dbg --config=asan @envoy//test/... \
+  bazel --batch test ${BAZEL_TEST_OPTIONS} -c dbg --config=clang-asan @envoy//test/... \
+    //:echo2_integration_test
+  exit 0
+elif [[ "$1" == "bazel.tsan" ]]; then
+  setup_clang_toolchain
+  echo "bazel TSAN debug build with tests..."
+  cd "${ENVOY_FILTER_EXAMPLE_SRCDIR}"
+  echo "Building and testing..."
+  bazel --batch test ${BAZEL_TEST_OPTIONS} -c dbg --config=clang-tsan @envoy//test/... \
     //:echo2_integration_test
   exit 0
 elif [[ "$1" == "bazel.dev" ]]; then
+  setup_clang_toolchain
   # This doesn't go into CI but is available for developer convenience.
   echo "bazel fastbuild build with tests..."
   cd "${ENVOY_CI_DIR}"
@@ -51,6 +62,7 @@ elif [[ "$1" == "bazel.dev" ]]; then
   bazel --batch test ${BAZEL_TEST_OPTIONS} -c fastbuild //test/...
   exit 0
 elif [[ "$1" == "bazel.coverage" ]]; then
+  setup_gcc_toolchain
   echo "bazel coverage build with tests..."
   export GCOVR="/thirdparty/gcovr/scripts/gcovr"
   export GCOVR_DIR="${ENVOY_BUILD_DIR}/bazel-envoy"

test/main.cc

@@ -5,6 +5,11 @@
 #include "exe/signal_action.h"
 #endif
 
+const char* __asan_default_options() {
+  static char result[] = {"check_initialization_order=true strict_init_order=true"};
+  return result;
+}
+
 // The main entry point (and the rest of this file) should have no logic in it,
 // this allows overriding by site specific versions of main.cc.
 int main(int argc, char** argv) {

tools/bazel.rc

@@ -2,10 +2,42 @@
 
 build --workspace_status_command=bazel/get_workspace_status
 
-build:asan --copt -fsanitize=address
-build:asan --linkopt -fsanitize=address
+# Basic ASAN/UBSAN that works for gcc-4.9
+build:asan --copt -fsanitize=address,undefined
+build:asan --linkopt -fsanitize=address,undefined
 build:asan --linkopt -ldl
 build:asan --define tcmalloc=disabled
 build:asan --build_tag_filters=-no_asan
 build:asan --test_tag_filters=-no_asan
 build:asan --define signal_trace=disabled
+
+# Clang 5.0 ASAN
+build:clang-asan --copt -D__SANITIZE_ADDRESS__
+build:clang-asan --copt -fsanitize=address,undefined
+build:clang-asan --linkopt -fsanitize=address,undefined
+build:clang-asan --copt -fno-sanitize=vptr
+build:clang-asan --linkopt -fno-sanitize=vptr
+build:clang-asan --copt -fno-sanitize=vptr
+# TODO(htuch): We should cleanup unaligned pointers at some point.
+# https://github.com/lyft/envoy/issues/914
+build:clang-asan --linkopt -fno-sanitize=alignment
+build:clang-asan --copt -fno-sanitize=alignment
+build:clang-asan --linkopt -fno-sanitize=vptr
+build:clang-asan --copt -fno-sanitize-recover=all
+build:clang-asan --linkopt -ldl
+build:clang-asan --define tcmalloc=disabled
+build:clang-asan --build_tag_filters=-no_asan
+build:clang-asan --test_tag_filters=-no_asan
+build:clang-asan --define signal_trace=disabled
+
+# Clang 5.0 TSAN
+build:clang-tsan --copt -fsanitize=thread
+build:clang-tsan --linkopt -fsanitize=thread
+build:clang-tsan --define tcmalloc=disabled
+
+# Clang 5.0 MSAN - broken today since we need to rebuild lib[std]c++ and external deps with MSAN
+# support (see https://github.com/lyft/envoy/issues/443).
+build:clang-msan --copt -fsanitize=memory
+build:clang-msan --linkopt -fsanitize=memory
+build:clang-msan --define tcmalloc=disabled
+build:clang-msan --copt -fsanitize-memory-track-origins=2