//tests/failures seems broken

[aherrmann]

Describe the bug
run-tests -m '/failures/' doesn't seem to be working as expected. The test should attempt to build all targets under //tests/failures/... that are marked with the manual tag and expect them to fail. However, if I add the manual tag to a suceeding target, e.g. //tests/failures/transitive-deps:lib-c, then the overall test still suceeds.

To Reproduce
Apply the following patch and run bazel build //tests/run-tests && bazel-bin/tests/run-tests -m '/failures/'.

diff --git a/tests/failures/transitive-deps/BUILD.bazel b/tests/failures/transitive-deps/BUILD.bazel
index c2efbd9..c409602 100644
--- a/tests/failures/transitive-deps/BUILD.bazel
+++ b/tests/failures/transitive-deps/BUILD.bazel
@@ -42,6 +42,7 @@ haskell_library(
         ":lib-b",
         "//tests/hackage:base",
     ],
+    tags = ["manual"],
 )

 haskell_library(

Expected behavior
The test should fail with the above patch applied.

Environment

• OS name + version: openSUSE Tumbleweed 20190529
• Bazel version: 0.24.0 (shell.nix)
• Version of the rules: 5b0c9d9

Additional context
The following patch provides additional output

--- a/tests/RunTests.hs
+++ b/tests/RunTests.hs
@@ -143,6 +143,7 @@ assertSuccess = outputSatisfy (const True)
 assertFailure :: Process.CreateProcess -> IO ()
 assertFailure cmd = do
   (exitCode, stdout, stderr) <- Process.readCreateProcessWithExitCode cmd ""
+  print (exitCode, stdout, stderr)

   case exitCode of
     ExitFailure _ -> pure ()

The following patch resolves the initial problem.

diff --git a/tests/RunTests.hs b/tests/RunTests.hs
index 9cc83b6..a589457 100644
--- a/tests/RunTests.hs
+++ b/tests/RunTests.hs
@@ -63,7 +63,7 @@ main = hspec $ do

     for_ all_failure_tests $ \test -> do
       it test $ do
-        assertFailure (bazel ["build", "test"])
+        assertFailure (bazel ["build", test])

   -- Test that the repl still works if we shadow some Prelude functions
   it "repl name shadowing" $ do

However, it uncovers another failure

Failures:

  tests/RunTests.hs:65:7:
  1) failures //tests/failures/transitive-deps:lib-dFailure
       uncaught exception: IOException of type InvalidArgument
       fd:15: hGetContents: invalid argument (invalid byte sequence)

  To rerun use: --match "/failures///tests/failures/transitive-deps:lib-dFailure/"

  tests/RunTests.hs:65:7:
  2) failures //tests/failures/transitive-deps:lib-cFailure
       uncaught exception: IOException of type InvalidArgument
       fd:15: hGetContents: invalid argument (invalid byte sequence)

  To rerun use: --match "/failures///tests/failures/transitive-deps:lib-cFailure/"